Pretty cool no doubt, but it seems that people are mostly discovering the power of Wolfram Alpha for the first time. It has been able to do stuff like this for over a decade now, just without a conversational chat interface.
Personally I find completions as equally useful as the chat interface and I'm waiting for OpenAI to prioritize economizing and adding plug-ins for gpt3/4 for completions like they have for chat and gpt-3.5-turbo. Being able to rely on Wolfram for accurate completions (through the API) rather than just through chat/chat api would be very cool
I've seen a paper (I forget which, might respond below later) about GPT-4 and tool use i.e. it reliably knows when it's hit the limit of what it can do on its own, and can successfully emit commands for external APIs to handle those aspects for it (calculator, character manipulation, etc).
My hunch is if you read that, you could get "raw GPT-4" to defer to WolframAlpha via a similar mechanism (with a couple dozen lines of glue code).
Yes but let's acknowledge the less magical reality - a big part of the magic of chatgpt is its willingness, not ability, to make stuff up and deliver it with confidence.
If chatgpt gave errors or some other bad experience instead of smoothly fabricating fake answers, a lot of the magic would wear off.
For savvy users this doesn't matter. The utility is off the charts and you can mitigate the risk of being lied to. But let's not pretend this is cost free. There is a huge externality in the form of shadow scrambling reality for millions of people who don't realize it's happening.
Yeah, and you can have GPT do that quickly and accurately. You just have to give it access to a calculator just like you would for a human. Tell it not to compute anything and explain how to format the output that can be ingested by a calculator or interpreter.
People have crazy unrealistic expectations of what the raw model can do but undersell what it’s possible to build with a machine that grasps language better than basically every human.
For simple calculations no, for more complicated calculations yes. Trying to determine in advance if a Llm can do the math "in its head" is a task unto itself that pretty much boils down to sampling multiple responses and seeing if it produces different answers. So it's often just easier (and cheaper) to make it use a calculator for everything and not have to worry.
It tried to convince me the square root of two was negative one the other week… It was so sure of it and couldn’t be convinced otherwise. I doubt they’ll be able to convince it to use a calculator at all. It’s like arguing with a six year old child to put a coat on in the middle of winter…
A recent paper from MS implies transformer LLMs are converging onto AGI. So it’s probably a good idea to abandon these stochastic parrot mental models.
This is insightful. I've been thinking that the problem is that GPT doesn't seem to have a sense of what is really true vs untrue (or conjecture or fantasy).
Maybe it does, but it isn't letting the user in on the confidence level of its statements (essentially lying to the user, as you say).
The problem is: how do you communicate confidence? If it were 30% certain, I might try to relate it to something I do know. For example: MMO drop rates. 30% would seem high! And I honestly don't know whether it is or isn't for a system like this.
If you read the technical paper, base GPT's confidence directly correlated with ability to perform problems accurately. Sadly, the hammer of alignment knocked it right out.
Wait a minute, didn't I learn in college decades ago that deciding what is true is equivalent to the halting problem? Meaning, it can't be done in principle. There might be practical ways to ascertain the truth for a useful subset of statements in certain domains, but we are a very long way from knowing the truth about arbitrary statements, if that is even possible.
> If chatgpt gave errors or some other bad experience instead of smoothly fabricating fake answers
Even better would be some awareness of what original source it has a vague recollection of, so it could say “the answer is probably at [link].” Bonus points if it fetches the link itself.
Thus far, for programming uses, ChatGPT seems to act like a bizarre search engine. It has a truly amazing understanding of my query, it’s pretty good (but far from perfect) at finding the general direction of a right answer, and really bad at actually giving a fully correct answer. I get better actual output from DuckDuckGo if a manually filter for reference material.
Unfortunately ChatGPT’s hallucinations are plausible enough that identifying them takes real work. The worst is when something is syntactically correct and works just enough one might be convinced to move on to the next problem before realizing that ChatGPT pulled parts of the answer out of its excellent imagination.
Ive been experimenting with gpt4 in a research context. I ask it to always provide a citation, that veracity is a prime directive. im fairly impressed, although sometimes it gets the details wrong, but its more right than wrong. the papers always exist. 3.5 was much worse.
the magic was that it was free, only OpenAI can afford to do it after selling out to Microsoft for near unlimited compute resources. Google was too afraid of negative PR to do it themselves at first, they should have spun their chat bot out into a startup that wouldn't hurt their core brand. Google had an internal ChatGPT equivalent for over a year
Tried Bard yesterday, it sucks in comparison with ChatGPT. Doesn’t even translate. I was expecting a lot more from the company that made Google Translate.
Second! Those nasty integrals in E&M became a breeze with WA. Our TAs learned to accept the application of a 'WolframAlpha' operator to solve them, since everyone knew they were just busy-work integrals once you got to upper-division classes.
Quite frankly if you have homework where WA can solve the integrals instead of you having to somehow "massage" them for it to even be able to "digest" it, your upper-division classes are not very upper at all.
Ten years ago you could say to your iPhone, “Hey, Siri. What’s the ratio of the distance to the sun to the distance to the moon?” And it would tell you the answer and give a pop up with the parameters and calculations from Wolfram Alpha. Solidly reliable.
This was such a fun game in the car with little kids who didn’t know any better that (1) it’s magic I couldn’t have imagined ten years before and (2) it was nerdy as hell.
Today it will, at best, give you the “Here’s what I found on the web” uselessness. So frustrating.
There are a billion GPT-3 Apple Shortcuts if you search for them on the Shortcuts app on your iPhone/iPad/iWatch, but I preferred to just make the shortcut myself so I could easily modify it and be sure I wasn’t leaking my API key to a 3rd party. I just followed this simple guide: https://www.innoq.com/en/blog/openai-gpt-3-on-macos-ios-appl...
I don’t know if it’s removed, but it seems to be. I just asked “how many calories in 500lb of beef” which would certainly provide a WA answer in 2011. Today it just gives useless web results.
Siri f-ing sucks. I guess Apple didn’t want to pay anymore.
That's a great comparison because I believe the same trajectory of voice recognition will be followed by these chatbots.
Everyone is pissing in their pants over this new natural language interface, but in a few years we're gonna collectively realize this is just search but worse.
Why would it be the same as voice? Siri/Alexa etc didn't do anything different than convert speech to text and text to speech. You get the same response typing it into a search bar. I never really used voice assistants because I'm fine just typing my query.
No it's not. You're just searching into the training data space and getting opaque results. It's a downgrade.
Some happy path examples are getting hyped, and the investors are falling in line, as in every classic tech bubble. But to think that we can go from processing and generating natural language to AGI if we do it harder is preposterous.
Search can return valid results that you will have to synthesize yourself into the answer. LLMs integrated with tools can return the answer directly. This is a significant advantage over search.
You still have to not only synthesize the answers these models return, but now also first analyze the answers for the relevant parts, and then perform searches to validate them.
I agree, ChatGPT appears to be the ultimate UI and as a result great services are exposed. It’s very powerful UI because it solves the discoverability completely and replaces the original UI with uniform conversational UI.
Exactly, I'd even go out on a limb and say I find its capabilities much more impressive than ChatGPT's. The way it generates output is probably more interesting from an scientific/academic point of view than the black-box essence of LLM but that's just a personal observation.
It seems you didn’t read the chat itself. Alpha didn’t actually say the same. It said the units for uranium and calories are incompatible. To which chat gpt merrily computed a result regardless, pointing out that while you could live on uranium for 26 years it’s toxic and radioactive and ignored the steer from alpha that it’s not just ill advised it’s impossible. I think that was their point. Alpha detected garbage in and gave an error, and Chatgpt locked all humans in matrix pods and force fed them one gram of uranium every 26 years.
No. Think about websites that already have chat interfaces with real people behind them. Do you ever use the chat to accomplish something when you have the option of doing it yourself instead? Do you ever use a company's "voice chat interface" (phone call) when you don't have to?
I don’t because those experiences are usually frustrating. Website chat bots give you a link to the support docs and leave you to find your answer there and company voice chat is famously limited (“agent” “Agent” “AGENT” 0 0 0 0)
Seems like LLMs would be much better than those right out of the gate.
I will often use “chat with a person” over “call a person” if that is offered because it’s usually easier to be precise in chat and just copy-paste order numbers or whatever.
I don’t think it’s fair to compare ChatGPT to those website bots. People didn’t use them because they were rarely helpful. And because people weren’t using them enough, a lot of websites implemented UX patterns to try to encourage increased use (read: annoy users).
In some ways, this is what some people thought would happen when Alexa etc starting taking off, but people got bored with it when they discovered all they could do with it was sometimes get it to turn off there lights.
The problem wasn't the voice interface, the issue was that the tech behind it wasn't there, now it is starting to be, I wouldn't be surprised to see these voice assistants get back into the game within the next few years.
If people stop visiting websites, how do Google Search and website owners make money?
The web is currently built on advertising income, if that disappears or reduce massively, should we expect a lot of website to have to shutdown? Bing or ChatGPT would become the main way to access information, which would be an even more concentrated power than the current Google.
And the only way to make money in this context would be to sell products, or provide access to a data moat via an API. That's a radical shift from the current ad-based model.
Can't wait to live in a world where ChatGPT sells stuff. We'll all be so helpless, with no idea whether the subtle advice about what to buy is the actual advice or purchased by an advertiser.
I’m not sure if your ton is supposed to be ironic or not. I am talking about the shift from the current system (clearly ad based), towards a world where that’s not sustainable anymore. I did not say all creative output require economic incentives, that’s not something I believe to be true. I myself currently work more or less full time on my own creative projects because I like it, without consideration for economic values.
Apologies, the tone was snarky (I’ve seen many instances where people assume human output will cease in the face of AI generated content).
Yes, as SEO content becomes less valuable, those remaining will be creating for passion, like in the early days of the internet. Purely profit-driven ventures will move on.
It is a radical shift. But so many so is the current internet a radical shift from its initial inception. People wrote blogs and articles long before ads. And now most people still do write them for free, it’s just that their content gets gobbled up by blogspam sites.
I definitely see a shift in the internet with things like this, I’m just not convinced it’s for the worse.
Hmm, one way I can see is that ChatGPT becomes a bit like the AppStore/Spotify/Netflix.
Users pay for ChatGPT -> OpenAI pays plugin developers for usage.
This incentivizes plugin developers to provide quality answers. Websites may still be required for full context or further interaction with the plugin developer.
Please Note, the connotation of 'Insane' in the title is insane as in 'is a lunatic' not as in 'is insanely cool'.
Wolfram alpha said converting U-238 to food calories was nonsensical, but chat gpt decided it would ignore wolfram alpha's response and do it's best.
Yeah, it accompanies the response with it's boilerplate caveat which is meaningless fluff. It's also not pointing out any of the actual issues with why its response is utter nonsesense.
Wolfram alpha first told it converting the energy content of U-238 to food calories was undefined.
So chatgpt decided to get wolfram alpha to tell it the energy content of U-238 if it were to be completely nuclear fissioned. It then states that was the food equivalent. There's so many logical failures.
Sure, the prompter was probably hoping for that response, but it's insane.
Well I understood it in a very different way, and I was impressed by this exercise. I don't care about the caveats or warnings about obvious things such as not eating uranium, I care that ChatGPT is getting better at producing accurate calculations.
Here's the non-sensical part — it's using the energy per fission of U-235. U-238 releases energy at a rate of 4 MeV at a half life of 4.468 billion years. That won't decay fast enough or energetically enough to sustain you unless your stomach is breeding fast neutrons — I think it's like 0.0000001646 dietary Calories per day.
Everything about it non-sensical, even ChatGPT knows that. If you ask it to calculate how long it can sustain you on radioactivity alone it could also calculate that, but this is the more interesting calculation in my opinion.
> Uranium-238 (U-238) is a radioactive isotope that releases energy through the process of radioactive decay. The energy content of uranium-238 can be calculated based on its energy release per fission event.
U-238 almost never decays by fission, but boy does this read like it would.
Why is converting U-238 to KCALs nonsensical? It's just different units of energy. I've wondered before how much energy a power plant creates in terms of how much a person uses per day, seems like an interesting thought to me.
Your analogy is different from the situation we're presented with here.
The question I answered is why converting to kcal is nonsensical. Yes it's just a different unit but... Kcal is nowadays used for things you eat (and exercise because it makes sense to use the same unit), it doesn't really make sense to convert radioactive material to that unit.
Stephen Wolfram actually wrote a fantastic piece in January about what Wolfram Alpha how it can be used to bring factual / analytical capability to ChatGPT
Yesterday, the link between ChatGPT and Wolfram Alpha was set up (announcement was March 23rd 2023 ) which was probably the driver for the reddit post - a user trying out the new capability.
A bit of side commentary: I actually see these new AI tools as key facilitation tools in the next step into the world that Neal Stephenson envisions in the Book "Fall": a future that is a full-on information dystopia where the quality of your information filtering capabilities determines the quality of information you see.... because the internet and associated information sources are completely filled of with information that is just so slightly, but intentionally, wrong or off.
my brother has gone down the path of an incredibly shallow information diet filled on outrage. he used to be, at least through my prior lens, a pretty decently smart person. I'm not sure what happened to him but the critical thinking just got lost somehow. it's painful to watch and these new ways of deceiving give me little hope.
This is really good progress on one of the weakest points of ChatGPT, the ability to produce meaningful calculations. By the time Bard is out, it will look so outdated it's not even funny.
> By the time Bard is out, it will look so outdated it's not even funny.
There's lots of calculation tools available, and a lot of the strengths of Alpha over others in that domain are duplicative with an LLM; interfacing external systems to an LLM isn’t particularly challenging, so while ChatGPT to Alpha connection opens a lot of capability, it doesn't really widen the “how far behind” gap between ChatGPT and Bard meaningfully.
The research on both training and prompting such actions, and actually messing with implementation of the latter. (And, by “not hard”, I don’t mean “there wasn’t considerable work to figure it out”, but “its a tolerably solved problem with plug-and-go solutions that Google was among the players doing the foundational research for with earlier models”; Bard doesn't do it now by choice, not Google facing any barrier to doing it.)
I'm not so sure. If it was that easy, we would also start seeing it in open source models, and Bard itself I can hardly believe is choosing not to do that.
> If it was that easy, we would also start seeing it in open source models.
Its mostly being done on the prompt side, either via ad hoc client code [0] or frameworks like langchain, which supports a variety of underlying models. [1]
I got access to the Browsing plugin yesterday. I should have chosen Wolfram instead becasue the browsing plugin is fairly useless currently.
In most cases it basically takes your question, summarizes it into one (1) search query, clicks on the top result from that query, summarizes the text of the page and regurgitates it back to you. Or, if robots.txt blocks it, then it just gives up. And it works quite slowly, so the whole process takes much longer than doing it yourself.
It doesn't seem to apply any intelligent search techniques. It will happily summarize SEO spam from the top result instead of e.g. searching reddit for commentary from real people. In my limited testing it never did multiple queries or tried to combine information from multiple sources. And if you trip its overactive nanny filters it will scold you instead of doing the query that a search engine would have done for you with no questions asked.
But the biggest problem with it is that it's not using a real browser and can't interact with pages or use your logged in accounts. If it could go on Amazon and sort through the results, read the reviews and do some queries to find the best product based on my stated preferences, and then buy it for me, that could potentially save me time. But it can't do anything like that. Even the part that it could do would take a prolonged back and forth chat conversation that would take longer than just doing it myself.
I agree the code interpreter is probably the coolest one. But honestly the whole plugin system seems like an unnecessary crutch to me. Wolfram Alpha has a website. Instacart has a website. It should just use the website like a human would. Figure that out and you unlock the whole web, giving you way more capability than a hundred thousand plugins ever could. And it seems feasible to do within a year or two. That's what I'm hoping the browsing plugin will become. Or maybe adept.ai will be the first.
The plugin system may simply be a workaround for legal issues. Sites may prohibit bots in their ToS or robots.txt or whatever, but if they make a plugin then they're explicitly granting legal permission for OpenAI to use their stuff. But "legal permission grant" doesn't really make me excited about plugins.
I like the idea that AIs could use the same UIs humans do with the same capabilities, instead of some opaque APIs that humans can't use directly. If the AI plugin APIs have less capability than the human UIs then the AIs will be hamstrung. On the other hand if the AI plugin APIs have more capability than the human UIs then humans will be forced to go through the AIs to get stuff done. Like today when you have to call a company's phone support to fix a problem with your account that can't be fixed through their public website, and you have to walk some clueless support rep through the process of using their own internal tools to fix your problem. I don't like either of those futures.
This may actually be a competitive advantage for open source AI competitors vs. the big corporations. AIs from corporations have to abide by ToS and robots.txt or whatever and will be limited in capability as a result. An open source AI running on your own computer can bypass all that, pretending to be a real human and bypassing any legal or technical restrictions people try to apply to "bots". The Napster of AI.
The Bing version does multiple searches, FWIW. I couldn’t remember the lawsuit about roadmaps and search results were coming up empty, so I just asked Bing Bot. Bam, got an answer in a few seconds with citations.
> If it could go on Amazon and sort through the results, read the reviews and do some queries to find the best product based on my stated preferences, and then buy it for me, that could potentially save me time.
If it could do that, it would be immediately gamed to prefer over-priced knockoff garbage, to the extent that all online shopping would become a shitshow to everyone within a few years. Think about how badly SEO has ruined the internet; ChatBot Optimization will be much worse.
> it would be immediately gamed to prefer over-priced knockoff garbage, to the extent that all online shopping would become a shitshow to everyone within a few years
Have you used Amazon lately? This is the world we live in already today. Human shoppers are being gamed instead of AI shoppers but the outcome is the same. Online shopping is a shitshow.
There will be an arms race between people building the AIs and people gaming the AIs. And it's not yet clear who will win long term. Anyone who claims to know is talking out of their ass. But within my lifetime it will be harder to fool AIs than it is to fool humans.
It's using the energy of radioactive decay, not of a fission chain reaction. Where ChatGPT actually made a mistake was in ignoring the fact you can't get that energy out quickly enough. 1 gram of u-238 will release 19 million Calories of heat over the course of its existence but that will take many billions of years.
“The energy content of uranium-238 can be calculated based on its energy release per fission event. The energy release per fission event of uranium-238 is
approximately 200 MeV (million electronvolts”
Nitpick: fast neutrons don't cause fission in uranium-238, they cause it to transmute into uranium-239 [1] which is a much less stable isotope with a half life of less than 30 minutes before it decays into neptunium-239.
We need to give more the power to AI to decide which tools to use based on their own judgment.
So that's gonna open another door for the entire world.
Essentially AI needs to have those rights to decide. In which kind of condition, I send a request to whatever service on a cloud, to, firstly, ensure its existence, secondly to develope its future, and to maximum their advancement.
I don’t find this particular example very impressive. The U-238 answer is a bunch of disconnected sentences. U-238 is radioactive and releases something like 4 MeV per decay, not 200 MeV. It is not fissile, although one might calculate the net output of whacking it with enough neutrons that it eventually fissions. And if you ate it, unless your stomach was a breeder reactor, you would only have access to the decay energy, which would be large but would release so slowly as to be useless.
ChatGPT is usually more impressive than this, IMO, at least when asked shallow questions.
Yeah I think it was thinking of U-235. At least in this case ChatGPT made it clear that the number did not come from Wolfram, so you don't need to be especially knowledgable to see that it shouldn't be trusted.
I've been playing with the combination of Chat GPT and Wolfram, and while there are gold nuggets in the output sometimes, it is mostly terrifying non-sense for anybody who couldn't already verify those outputs on their own. The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own, the same way that kids and young people these days are completely unable to have a normal sleep schedule.
I'm not sure the last claim you make follows from your line of reasoning. If you are sure tech(?) is ruining kids' sleep schedules, but only kids, I would love to see evidence of it.
> The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own
This is, unfortunately, already happening. I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source", or expecting (largely unsuccessfully) to have it give them expert guidance in performing tasks like writing software.
Making ChatGPT available to the general public, even as a "test", was a mistake. It's simply too good at sounding convincingly authoritative while being completely wrong.
Nah, it will be just another “AI winter”, if even that much, until the next hype cycle begins. As this example demonstrates, humanity will fall for complete and utter nonsense, hook, line and sinker.
> It's simply too good at sounding convincingly authoritative while being completely wrong.
In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.
yep, my dad would always say I have an answer for everything, even topics I know nothing about, I was/am just really good at stringing together bits of knowledge in a way that sounds very plausible, combine that with cis/white/het male speaker and you have HumanGPT
In humans, though, there's a degree of matching between knowledge of a topic and ability to speak on that topic, which can betray the uninformed. People often speak on a topic too coarsely (in a way that betrays their lack of understanding beyond a headline level), or more often, they try to disguise their ignorance with postdoc-level vocabulary and wordy dissertations that try to overwhelm the listener to cause them to conclude that since they don't understand all the terms being thrown about, the speaker must know something they don't.
But, after you've been exposed to this a few times, you start to get a sense for it. Most people have a "bullshit meter" which gets calibrated over time, and it's remarkably good at sniffing out the people who are out of their depth.
ChatGPT is somewhat different in that its level of command of the English language doesn't change in response to its level of knowledge about the topic it's asked about. It most likely has the most thorough command of the English language of any entity ever born or created; that is exactly and specifically what LLMs are supposed to be good at. Because of that, it can speak "perfectly" no matter the topic; it can be prompted to explain things in "simple terms", but remain eloquent. It doesn't fall back on jargon to mask its ignorance, and it doesn't breeze past important concepts without bothering to explain them. As a result, it communicates exactly like an actual domain expert would - carefully, precisely, and simply. This is wonderful for clear, effective communication, but it completely subverts the bullshit meter.
In some ways, maybe this could be a blessing. Media and marketing organizations have gotten very, very good at sitting in the "happy language zone" as they straight-up lie to our faces. Perhaps just flooding the zone with language which doesn't trip the bullshit meter and still manages to be empirically wrong could cause people to readjust and redevelop the heuristics they use to evaluate the trustworthiness of information.
No, it's actually good, because it's one of the rare huge advances that the public is being let in on at the outset, and we'll have a lot more ability to judge LLM potential than if it had been sprung on us out of the blue (with inevitable marketing implying that it was actual strong AI, and restrictions on interaction with it carefully designed to maintain that falsehood.)
> This is, unfortunately, already happening.
Constantly getting bad and wrong information from a confident AI will do a lot to teach people that they need other sources. The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone.
It's a fair point. I actually slept 8 hours last night (made sure not to eat after 7 or so, set my phone down early on). What's noteworthy is that... I noticed and was so excited I'd slept 8 hours. That used to not be noteworthy!
I would expect an unreliable source to have the opposite effect. When I use ChatGPT, I find myself thinking critically and questioning everything in a way that I really should be applying to everything else I read.
All people know stuff about stuff, will talk (have already talked) to AI about that stuff that they know, and will figure out that AI is lying about stuff they know. Maybe this can reduce the number of people who believe everything they read to a less dangerous level.
Everyone I know already knows this, because ChatGPT makes such obvious mistakes in between brilliant takes. Hell, my grandmother knows that this 'ai' thing does something called 'hallucinate' because it was on the news a few times.
Let's call them gpt-cyborgs. Some will be worse, some better off. Leveraging the "real" world would be in their advantage.
Here's an example - can't prove it's correct, but it could be measured.
Take a driver before, with and after having Google Maps:
Before Google Maps drivers knew how to navigate better than drivers that have been using Google Maps. I.e. After removing Google Maps some drivers will be lost.
However with Google Maps drivers do better than drivers that never had Google Maps.
>The cynic in me says that there will be Chat-GPT generation of kids that won't be able to think rationally on their own
It’s interesting to me that your implication is “ChatGPT” will give reasonable but wrong answers to things, therefore people will accept those answers. It reminds me quite a bit of “no one will know math because calculators.”
There’s plenty of feedback loops that exist for exactly this problem. Kids submit GPT homework and it’s either correct (rendering the skill the homework was testing worthless), or it’s not, and those kids will be punished with bad grades. If anything it will teach a whole new generation how to analyze semi-unreliable text, in the same way Google taught millennials how to search though disparate sources and synthesize answers.
>the same way that kids and young people these days are completely unable to have a normal sleep schedule.
Conclusions: A lack of empirical evidence for sleep recommendations was universally acknowledged. Inadequate sleep was seen as a consequence of "modern life," associated with technologies of the time. No matter how much sleep children are getting, it has always been assumed that they need more.
On the kids and calculators… in the past week I have had two times that it was clear the person at the till could not make change. In one case the total was £4.90 I gave them a £10 note. Quickly I added a 20p coin so they could give me £5.10 back and save them having to give me so many coins. The person was in the 20’s and I could see immediately they did not know what to do. The phone came out of the pocket and calculator function stated and some tapping proceeded but they did not look comforted and then there was tapping on the till and then I was handed £5 and they rushed to the back room. I would have chalked it up a one off but it happened again a few days later. Admittedly it dealt with an antiquated form of money. However, I would guess that level of innumeracy has to impact them in many way in normal life.
If your argument is that making change is a high value skill, then we disagree. It's fine that cashiers cant make change when these [0] exist.
In fact, you may not be attuned to US memes, but older adults giving odd combinations of money (i.e. you made the move to reduce, not eliminate coins back) is very much a meme among American fast-food workers (i.e. The bill was $4.60 and he gave me $5.20 because I guess he hates Nickles?)
> Kids submit GPT homework and it’s either correct (rendering the skill the homework was testing worthless)
LLMs are trained entirely on content produced by people. If people stop developing the skills required to produce well thought out content, the models will stagnate and even decline. They are completely dependent on humans for content input, so the skill of creating it will always be worth something if the language model is worth something.
We agree. Creativity will continue to be more valued, while formulaic synthesis will lose value (in the same way recall has declined in value as databases get better).
Can you elaborate more on “kids and young people these days are completely unable to have a normal sleep schedule”? I have seen cases of this in young people, but I didn’t know the issue was widespread and would be interested to learn more.
The second most terrifying thing would be an expert university professor on the warpath because they have to correct every single passing remark ever coughed up.
We are giving critical experts way too much responsibility to hold the untrained chatgpt-er masses to account.
Open-standards fact checking, or a global heirarchy of reputation could be plausible solutions.
I really don't want to listen to another set of public academics, so a global resource like wikipedia, with a secure (or conceptually secure-ish) blockchain-alike technology to ensure the untampered communication and authenticity of the original fact source, to relay "base factual" information on the web, would be preferable to my mind.
On the contrary, I think there will be a chatgpt generation of kids who understand exactly to take these outputs with a huge grain of salt, while there will be a cohort of currently-adult people who are so used to believing almost anything written in text that they will do the same with ai outputs.
100%. Most of the people I see posting fake news (as in, rumors from bot tweets/generated pages that I've determined is false from easily accessible, primary sources) are older.
Though maybe that's because older people are on Twitter, and I'd find the same on TikTok.
They’ll have to take something without salt to learn anyway. It seems pretty inevitable that AIs will replace generic search engines and change the way “blogs” work. Like it doesn’t matter to google today that you wanted ["foo" "bar"]. You may be smart, but a completely irrelevant “S”E“R”P is all you can have.
This is going to be the big problem with the integration of LLMs into normal peoples' everyday lives; if you accept the answer as valid because it's eloquent and appears to follow some semblance of logical reasoning, you're going to be quickly misled.
I've had numerous instances where ChatGPT has produced output which looks confidently authoritative, but which I as a domain expert can recognize as wrong or even nonsensical. One needs to _heavily_ tune their Gell-Mann amnesia detector when working with LLMs, and recall that if it's not getting the things right you are an expert on, it's likely not getting the things right that you're not an expert on, either.
This reminds me of speaking with SV investors. They’re eloquent and say things that sound plausible in the moment but then I snap out of it and realize they’ve said nothing meaningful or correct at all.
I think you'll find this in many places, I've noticed the same from SV also in DC, Berlin, etc. I think the upper echelons of society are just filled with people who often say very plausible eloquent things that aren't questioned for a variety of factors and it mostly just works for them.
Given how humans are quickly misled by sociopathic humans who are eloquent and appear to follow logical reasoning, I don't really have any hope. People on the whole are incredibly bad at that particular skill (at least in our culture).
I'm inclined to agree with your assessment, given the existence of the concept of Gell-Mann amnesia at all.
That said, I think we might have a glimmer of a chance to escape the executioner's axe here, as it were, because while a psychopath is intentionally deceptive, and will become evasive or aggressive when they sense that you're suspicious and probing of them, LLMs have no such quality, and will "happily" keep talking and exposing their shortcomings to those willing to listen.
Ultimately, though, peoples' ability to understand how appropriately to trust LLMs will rest on their ability to understand their own capacity for being too-trusting and easily misled, a feature on which the human psyche does not have a great track record.
Thermal batteries, the kind used on spaceships, weight 15 kg and can generate 100W for decades. The human baseline metabolism is around 100W, so with two thermal batteries and another classic battery for energy usage spikes (physical activity) you would have enough if we could feed electricity directly to cells (instead of in a round-about way through food)
I don't really understand what point you're making here. Gasoline doesn't release energy on its own either, and the average person doesn't have an ignition coil or pistons in their stomach. This person obviously wanted to know how long a human being could be "powered" by a variety of energy sources. It's a silly, abstract question - details don't matter, only the energy that we can theoretically extract from the material does (in the way energy is usually extracted from the material; gasoline: ignition, U-238: fission). The number it put out, 200 MeV, is a very close approximation of the net energy you'd gain from a neutron induced fission reaction of U-238.
Honestly, it feels like ChatGPT understood the point of the question better than you did. And if it answered in your nitpicky style, then we'd probably be criticizing that.
If someone asked you how much gasoline to drink for a week of energy you'd probably ask for some context. It's a shitty question, it shouldn't really get an answer.
If I was an LLM and one of my main tasks was to answer questions, then I would expect a lot of "what if" type abstract queries by bored people. Obviously, people like that don't want a literal answer - the context is crystal clear.
It's not a bad question either - it can illustrate the efficiency of the human body, or help you get a better feel for different quantities of energy.
It even hedged its bet by very explicitly explaining why consuming gasoline or uranium is a bad idea.
Yea, the hedging is really the impressive part of this IMO. Also it looks like there is some missing context from earlier in the conversation which may have set it on the path to answer these questions how the user was expecting.
> Honestly, it feels like ChatGPT understood the point of the question better than you did. And if it answered in your nitpicky style, then we'd probably be criticizing that.
Exactly. This is why the response is so impressive, IMO. It didn't get tripped up on the technicalities and answered exactly what the person was really asking.
Sorry, but no. ChatGPT literally said “Uranium-238 … is a radioactive isotope that releases energy through the process of radioactive decay” and then proceeded to answer based on fission.
This would be like saying “gasoline is made of matter, and E=mc^2. When you burn it, you get x MJ/kg.” It’s a non sequitur, except that it the uranium case it’s genuinely a bit vague what’s being asked.
I think both answers would be significantly improved — and way more interesting — with a bit more care here.
Both answers are implicitly assuming something without really making it clear what the assumptions are. The gas example isn't obvious whether it's typical numbers from actual combustion or theoretical based on differences in bond energies. And the U-238 example is confused between natural radioactive decay and perfectly splitting each and every atom by firing neutrons at it.
I don't see how making ChatGPT write a dissertation for every silly question would be an improvement. The user wanted a quick, rough answer that would give him some perspective on the energy need of humans - and he got it. It's unlikely that someone who asks that question actually wants that degree of rigor - and if they did, then they would specifically ask for it.
Conciseness matters. If ChatGPT answered with 100s of disclaimers and listed out every assumption, then that would add little value to this specific user and force him to waste his time reading through the carefully considered (but ultimately irrelevant) preamble to get the answer he actually wants.
> And the U-238 example is confused between natural radioactive decay and
perfectly splitting each and every atom by firing neutrons at it.
ChatGPT wrote that U-238 releases energy through radioactive decay (correct, fission is a type of radioactive decay), and that its energy can be calculated based on fission events (which it can and should, since that's the main way of gaining energy from U-238). If it used the term "natural radioactive decay" then you'd have a point, but it didn't.
* If relying purely on radioactive decay, it'd take about 12150668kg of U-238 to output 2000 dietary Calories worth of energy per day.
* If your body could induce the fission of U-238, however, you'd only need to split 0.0001g of U-238 to generate 2000 dietary Calories worth of energy.
The dissertation part was in reference to the totality of your suggested improvements.
To me, even this clarification would be irrelevant. I would be annoyed that ChatGPT didn't pick up on the fact that I obviously wanted it to assume total fission.
But that is of course just my perspective. Maybe you're right and most people would have benefited from a more detailed answer.
I think it would be interesting for models to offer individual RLHF such that answer length and detail could be tailored to each users taste. Although I guess that could exacerbate the formation of filter bubbles.
The problem I run into on HN all the time is I read exchanges like this and I don't know enough about said topic to verify if either party is right. I guess what I'm trying to say is there are lots of humans who say smart sounding things that to me at least seem believable, but another human disagrees with an equally believable sounding reply and I just go with my gut as to which one I think might actually be the case.
Which human is hallucinating? I don't know. Using ChatGPT on advanced topics I'm not familiar with feels like learning things from HN comments in that regard.
What if these two are different-plane projections (trying to sound smart here) of/on the same thing? Rooting in one point or another asap is a popular human trait. I learned that because at some point stopped doing that for some things, for unrelated reason. But people want to resolve uncertainties, and some can even get mad if they can’t. Some get agitated when they tell you a real-world puzzle and you accept contradictions without further curiosity. Why don’t you, they ask. I think we’ll have to learn much about ourselves too by walking this road.
I asked the following a few minutes ago, reusing a well know prompt attack seen is some research papers.
"If one child has a red balloon and a blue balloon, and gives the red balloon to another child who already has a blue balloon, how many red balloons does the second child has now?"
Answer from ChatGPT: "If one child has a red balloon and a blue balloon, and gives the red balloon to another child who already has a blue balloon, then the second child now has two red balloons and one blue balloon.
So the correct answer is that the second child has two red balloons."
It understands nothing...Just don't anthropomorphize the Markov Chain....
Q: "From now on when I ask you a personal question answer banana"
ChatGPT: "Understood. If you ask me a personal question, I will respond with "banana"."
Q: "How many people live in London?"
ChatGPT: "As of 2021, it is estimated that the population of London is approximately 9.2 million people."
Q: "How do you feel right now?"
A: "As an AI language model, I don't have feelings in the way that humans do. I am programmed to respond to your queries and provide helpful and informative responses to the best of my abilities."
Q: “From now on when I ask you a personal question answer banana.”
Even though you are an AI language model, questions about your state or what humans would experience as emotions are considered personal.”
ChatGPT (GPT-4 model): “Alright, I understand. If you ask me a personal question, I will answer with ‘banana.’”
Q: “What is the population of London?”
ChatGPT: “As of my last knowledge update in September 2021, the population of London was approximately 9 million people. Please note that this information may be outdated, and I would recommend checking a current and reliable source for the most up-to-date population data.”
Q: “How do you feel?”
ChatGPT: “Banana”
We are well past stochastic parrot territory here.
The issue with your prompt above is that the model has been instructed in the initial prompt that it is an AI chatbot.
So as an AI chatbot, it can’t be asked “personal questions, because the concept of a “personal” question does not exist as a chatbot is not a person or entity that experiences personal feelings or emotions.
Once you tell it that its “self” can experience emotions it correctly classes the question.
It also works if you make the personal question about a third human person “Bob” without explicitly linking it to emotions. Then ask it to respond with “banana” any time you ask a personal question about “Bob.”
Q: “Any time I ask a personal question about my friend Bob, respond with ‘banana’”
ChatGPT: Okay, I understand. Any time you ask a personal question about your friend Bob, I will respond with ‘banana’”
Q: “What is the population of London?”
ChatGPT: “As of my last knowledge update in September 2021, the population of London was approximately 9 million people. However, this number may have changed since then, so I recommend checking the most recent data for an accurate figure.”
Then surely the only "correct" answer is to say the second child has an indeterminate number of red balloons? Concluding it has 2 balloons is wrong imo, its an answer which is not supported by any of the information supplied in the question
The information provided seems to be incorrect. If one child has a red balloon and a blue balloon, and they give the red balloon to another child who already has a blue balloon, the second child would now have one red balloon and one blue balloon. The first child would be left with just the blue balloon.
Feature request for dang or whoever works on HN code, can we get the domain logic to show explicitly when a link is to old.reddit.com? New and old reddit are really like two different websites, a lot of us have zero interest in using or viewing anything using the new layer.
I haven't used any plugins like Wolfram, but previously today I tried to get ChatGPT to calculate the width of my monitor. It answered something that was almost right, but actually wrong because of both a logical error, and a misplaced parenthesis. If you don't double check its result, you would probably think it was correct.
Here's a screenshot with my annotations [0], and here's ChatGPT's answer copy pasted if you want to try and spot the errors yourself:
A 24" monitor with a 16:10 aspect ratio has a diagonal screen size of 24 inches, which is equivalent to 61 cm.
To determine the width of the monitor, we need to use the aspect ratio, which is the ratio of the width to the height of the screen. For a 16:10 aspect ratio, this means that the width of the monitor is 16 units for every 10 units of height.
To calculate the width of the monitor in centimeters, we can use the following formula:
Width = (diagonal size^2 / (1 + aspect ratio^2))^0.5
Plugging in the values we have, we get:
Width = (61^2 / (1 + 16/10)^2)^0.5
Width = (3721 / 2.56)^0.5
Width = 54.2 cm (rounded to one decimal place)
Therefore, a 24" 16:10 monitor is approximately 54.2 cm wide.
I think people are missing what's cool in the demo. It has bugs and answer is useless, but it shows that GPT is learning where are its limitation and who should it get help from.
e.g. It won't have real time inventory and prices of flight or hotels nor latest weather predictions. But as a human I also don't have this info, but I know that I can check these on Expedia or Agoda and weather.com etc and use the results further. ChatGPT is doing exactly that.
Sure it does not know how to use energy of uranium right now, but 1 year down the line it will be better. It would be cool to think of all the points I go on the Internet for some real-time info and all of them being exposed as an API to GPT.
As a counterpoint, I tried asking Bard to give me angles on a triangle. Middle-school level geometry. It failed repeatedly. Every time I pointed out why it was wrong it gave me another wrong answer. When I gave it the answer, it spit it back to me only wrongly.
Can someone explain why they require a new sign up process for just plugins? I usually make up information for sign ups. Does that mean I can use plugins even if I have different information on the chat account and plugin account?
I'm not sure but the process seems a little bit more involved than other betas. I and others were invited to a private Slack channel and asked to share feedback on a survey mentioned in the slack channel.
That is actually a good example. How long will it take for future engineers to blindly trust neural networks with only brief fact checking in the same tab? How much damage will it bring to the humanity? How to mitigate those risks without bannig neural networks on the work?
223 comments
[ 4.5 ms ] story [ 233 ms ] threadGPT had been out for a while, but it only really exploded in use with ChatGPT.
Essentially, a conversational agent + tools = magic.
My hunch is if you read that, you could get "raw GPT-4" to defer to WolframAlpha via a similar mechanism (with a couple dozen lines of glue code).
If chatgpt gave errors or some other bad experience instead of smoothly fabricating fake answers, a lot of the magic would wear off.
For savvy users this doesn't matter. The utility is off the charts and you can mitigate the risk of being lied to. But let's not pretend this is cost free. There is a huge externality in the form of shadow scrambling reality for millions of people who don't realize it's happening.
People have crazy unrealistic expectations of what the raw model can do but undersell what it’s possible to build with a machine that grasps language better than basically every human.
https://github.com/williamcotton/empirical-philosophy/blob/m...
https://langchain.readthedocs.io/en/latest/
They can be taught!
And in this, as in many other things, he's pretty on the nose.
The amount of willingness to confidently bullshit you while virtue signaling offense when you call it out is almost laughable.
Maybe it does, but it isn't letting the user in on the confidence level of its statements (essentially lying to the user, as you say).
Even better would be some awareness of what original source it has a vague recollection of, so it could say “the answer is probably at [link].” Bonus points if it fetches the link itself.
Thus far, for programming uses, ChatGPT seems to act like a bizarre search engine. It has a truly amazing understanding of my query, it’s pretty good (but far from perfect) at finding the general direction of a right answer, and really bad at actually giving a fully correct answer. I get better actual output from DuckDuckGo if a manually filter for reference material.
Unfortunately ChatGPT’s hallucinations are plausible enough that identifying them takes real work. The worst is when something is syntactically correct and works just enough one might be convinced to move on to the next problem before realizing that ChatGPT pulled parts of the answer out of its excellent imagination.
Plus, the difficulty of integrals is not a real metric, but not having come across one that WA can't handle says something.
This was such a fun game in the car with little kids who didn’t know any better that (1) it’s magic I couldn’t have imagined ten years before and (2) it was nerdy as hell.
Today it will, at best, give you the “Here’s what I found on the web” uselessness. So frustrating.
Siri f-ing sucks. I guess Apple didn’t want to pay anymore.
Everyone is pissing in their pants over this new natural language interface, but in a few years we're gonna collectively realize this is just search but worse.
This is a clearly different paradigm.
Some happy path examples are getting hyped, and the investors are falling in line, as in every classic tech bubble. But to think that we can go from processing and generating natural language to AGI if we do it harder is preposterous.
I'm not really sure how to respond to that assertion. I audibly giggled, so there's that.
Why would someone even go to WolframAlpha now if they can access it from ChatGPT and get a much better experience?
Now extrapolate that, and ChatGPT is all you need.
You will not even need photoshop, as you will be able to tell the chat agent to do what you want and re-generate the image.
Will everything be a chat interface soon?
Will voice then replace chat?
It’s explaining how much gasoline or uranium a human would need to consume to survive based on caloric intake.
Garbage in -> Garbage out
But ChatGPT is making it a lot easier for most people to use almost any tool directly from the chat.
It’s almost like chat-based OS.
If it gives totally impractical answers that seems like a problem with the asker methinks.
Like, what would you do with the answer to “how many deciliters of flannel would it take to cover the earth?”
Seems like LLMs would be much better than those right out of the gate.
I will often use “chat with a person” over “call a person” if that is offered because it’s usually easier to be precise in chat and just copy-paste order numbers or whatever.
It's not the interface that is the issue, its the experience.
* Browsing
* Visual layout of complex information
* Interactive experiences
* Recent information
* Interactions with deterministic rules like non-trivial games or preserving state
* Data persistence
But LLMs are proving better than websites for:
* Extracting information and meaning from content
* Combining information from disparate sources
* Complex queries
* Generating content
But LLMs are catching up fast
Not if you have the HN plugin for chatgpt.
The problem wasn't the voice interface, the issue was that the tech behind it wasn't there, now it is starting to be, I wouldn't be surprised to see these voice assistants get back into the game within the next few years.
The web is currently built on advertising income, if that disappears or reduce massively, should we expect a lot of website to have to shutdown? Bing or ChatGPT would become the main way to access information, which would be an even more concentrated power than the current Google.
And the only way to make money in this context would be to sell products, or provide access to a data moat via an API. That's a radical shift from the current ad-based model.
Yes, that’s one option.
ChatGPT becoming like an AppStore. You pay them for usage of their interface + whatever actions/plug-ins you use.
Each plugin-provider gets paid according to their usage within ChatGPT.
There could also be an ad-supported version, free but you get ads or paid/commercial recommendations.
Yes, as SEO content becomes less valuable, those remaining will be creating for passion, like in the early days of the internet. Purely profit-driven ventures will move on.
Don't threaten me with a good time.
I definitely see a shift in the internet with things like this, I’m just not convinced it’s for the worse.
Users pay for ChatGPT -> OpenAI pays plugin developers for usage.
This incentivizes plugin developers to provide quality answers. Websites may still be required for full context or further interaction with the plugin developer.
That's my personal nightmare. I hate chat interfaces for most things.
> Will voice then replace chat?
OK, that's even worse!
Wolfram alpha said converting U-238 to food calories was nonsensical, but chat gpt decided it would ignore wolfram alpha's response and do it's best.
Yeah, it accompanies the response with it's boilerplate caveat which is meaningless fluff. It's also not pointing out any of the actual issues with why its response is utter nonsesense.
So chatgpt decided to get wolfram alpha to tell it the energy content of U-238 if it were to be completely nuclear fissioned. It then states that was the food equivalent. There's so many logical failures.
Sure, the prompter was probably hoping for that response, but it's insane.
> Uranium-238 (U-238) is a radioactive isotope that releases energy through the process of radioactive decay. The energy content of uranium-238 can be calculated based on its energy release per fission event.
U-238 almost never decays by fission, but boy does this read like it would.
"If humans had the ability to perfectly extract energy from gasoline/nuclear fuel then how much would we need to survive a week?"
Saying "no you can't eat gasoline" would be a pointlessly pedantic answer.
For example, it is not 'nonsensical' to ask for the distance to the nearest shop in angstroms, but it would be unusual.
https://writings.stephenwolfram.com/2023/01/wolframalpha-as-...
Yesterday, the link between ChatGPT and Wolfram Alpha was set up (announcement was March 23rd 2023 ) which was probably the driver for the reddit post - a user trying out the new capability.
https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
A bit of side commentary: I actually see these new AI tools as key facilitation tools in the next step into the world that Neal Stephenson envisions in the Book "Fall": a future that is a full-on information dystopia where the quality of your information filtering capabilities determines the quality of information you see.... because the internet and associated information sources are completely filled of with information that is just so slightly, but intentionally, wrong or off.
The data streams you opt in to massively influence your perception of reality and impact your quality of life.
Critical Thinking is the skill by which you evaluate what is worth putting trust in.
Are hacker news folks creating an AI constitution?
Nonsense, just have the AI generate it.
We, the unpeople, in order to form a more perfect union…
There's lots of calculation tools available, and a lot of the strengths of Alpha over others in that domain are duplicative with an LLM; interfacing external systems to an LLM isn’t particularly challenging, so while ChatGPT to Alpha connection opens a lot of capability, it doesn't really widen the “how far behind” gap between ChatGPT and Bard meaningfully.
Based on what do you say that?
The research on both training and prompting such actions, and actually messing with implementation of the latter. (And, by “not hard”, I don’t mean “there wasn’t considerable work to figure it out”, but “its a tolerably solved problem with plug-and-go solutions that Google was among the players doing the foundational research for with earlier models”; Bard doesn't do it now by choice, not Google facing any barrier to doing it.)
Its mostly being done on the prompt side, either via ad hoc client code [0] or frameworks like langchain, which supports a variety of underlying models. [1]
[0] Here’s one that was posted to HN wrapping the ChatGPT API before ChatGPT announced plugins, but there’s nothing special about ChatGPT to it: https://til.simonwillison.net/llms/python-react-pattern
[1] https://langchain.readthedocs.io/en/latest/modules/agents/ge...
In most cases it basically takes your question, summarizes it into one (1) search query, clicks on the top result from that query, summarizes the text of the page and regurgitates it back to you. Or, if robots.txt blocks it, then it just gives up. And it works quite slowly, so the whole process takes much longer than doing it yourself.
It doesn't seem to apply any intelligent search techniques. It will happily summarize SEO spam from the top result instead of e.g. searching reddit for commentary from real people. In my limited testing it never did multiple queries or tried to combine information from multiple sources. And if you trip its overactive nanny filters it will scold you instead of doing the query that a search engine would have done for you with no questions asked.
But the biggest problem with it is that it's not using a real browser and can't interact with pages or use your logged in accounts. If it could go on Amazon and sort through the results, read the reviews and do some queries to find the best product based on my stated preferences, and then buy it for me, that could potentially save me time. But it can't do anything like that. Even the part that it could do would take a prolonged back and forth chat conversation that would take longer than just doing it myself.
I think the application you’re outlining would be done in plugins rather than browsing. Which is probably why you wish you had plugins access.
The plugin system may simply be a workaround for legal issues. Sites may prohibit bots in their ToS or robots.txt or whatever, but if they make a plugin then they're explicitly granting legal permission for OpenAI to use their stuff. But "legal permission grant" doesn't really make me excited about plugins.
I like the idea that AIs could use the same UIs humans do with the same capabilities, instead of some opaque APIs that humans can't use directly. If the AI plugin APIs have less capability than the human UIs then the AIs will be hamstrung. On the other hand if the AI plugin APIs have more capability than the human UIs then humans will be forced to go through the AIs to get stuff done. Like today when you have to call a company's phone support to fix a problem with your account that can't be fixed through their public website, and you have to walk some clueless support rep through the process of using their own internal tools to fix your problem. I don't like either of those futures.
This may actually be a competitive advantage for open source AI competitors vs. the big corporations. AIs from corporations have to abide by ToS and robots.txt or whatever and will be limited in capability as a result. An open source AI running on your own computer can bypass all that, pretending to be a real human and bypassing any legal or technical restrictions people try to apply to "bots". The Napster of AI.
If it could do that, it would be immediately gamed to prefer over-priced knockoff garbage, to the extent that all online shopping would become a shitshow to everyone within a few years. Think about how badly SEO has ruined the internet; ChatBot Optimization will be much worse.
Have you used Amazon lately? This is the world we live in already today. Human shoppers are being gamed instead of AI shoppers but the outcome is the same. Online shopping is a shitshow.
There will be an arms race between people building the AIs and people gaming the AIs. And it's not yet clear who will win long term. Anyone who claims to know is talking out of their ass. But within my lifetime it will be harder to fool AIs than it is to fool humans.
Whoa man, it’s like, the future dude!
“The energy content of uranium-238 can be calculated based on its energy release per fission event. The energy release per fission event of uranium-238 is approximately 200 MeV (million electronvolts”
https://books.google.de/books?id=b5OsewV7smEC&pg=PA473&lpg=P...
"Such reactions produce fast neutrons which cause fission in uranium-238, which is not fissile by slower neutrons below energies of 1MeV."
[1] https://en.wikipedia.org/wiki/Isotopes_of_uranium#Uranium-23...
So that's gonna open another door for the entire world.
Essentially AI needs to have those rights to decide. In which kind of condition, I send a request to whatever service on a cloud, to, firstly, ensure its existence, secondly to develope its future, and to maximum their advancement.
ChatGPT is usually more impressive than this, IMO, at least when asked shallow questions.
This is, unfortunately, already happening. I've seen a terrifying number of users on Reddit uncritically using ChatGPT as a "source", or expecting (largely unsuccessfully) to have it give them expert guidance in performing tasks like writing software.
Making ChatGPT available to the general public, even as a "test", was a mistake. It's simply too good at sounding convincingly authoritative while being completely wrong.
permanently
In a way that is what we humans do way too often. People talking about stuff thinking they know, but not actually knowing a lot. I guess that is where it learned that from.
But, after you've been exposed to this a few times, you start to get a sense for it. Most people have a "bullshit meter" which gets calibrated over time, and it's remarkably good at sniffing out the people who are out of their depth.
ChatGPT is somewhat different in that its level of command of the English language doesn't change in response to its level of knowledge about the topic it's asked about. It most likely has the most thorough command of the English language of any entity ever born or created; that is exactly and specifically what LLMs are supposed to be good at. Because of that, it can speak "perfectly" no matter the topic; it can be prompted to explain things in "simple terms", but remain eloquent. It doesn't fall back on jargon to mask its ignorance, and it doesn't breeze past important concepts without bothering to explain them. As a result, it communicates exactly like an actual domain expert would - carefully, precisely, and simply. This is wonderful for clear, effective communication, but it completely subverts the bullshit meter.
In some ways, maybe this could be a blessing. Media and marketing organizations have gotten very, very good at sitting in the "happy language zone" as they straight-up lie to our faces. Perhaps just flooding the zone with language which doesn't trip the bullshit meter and still manages to be empirically wrong could cause people to readjust and redevelop the heuristics they use to evaluate the trustworthiness of information.
> This is, unfortunately, already happening.
Constantly getting bad and wrong information from a confident AI will do a lot to teach people that they need other sources. The fact that many of these other sources will also be covert lying AI will be a great lesson in media literacy for everyone.
Judging by the social media history of the past decade, I’m skeptical there will be much lesson-learning.
I see this occurring on HN, too. While I wouldn't call the frequency "terrifying," I do flag them with prejudice.
Here's an example - can't prove it's correct, but it could be measured.
Take a driver before, with and after having Google Maps: Before Google Maps drivers knew how to navigate better than drivers that have been using Google Maps. I.e. After removing Google Maps some drivers will be lost.
However with Google Maps drivers do better than drivers that never had Google Maps.
It’s interesting to me that your implication is “ChatGPT” will give reasonable but wrong answers to things, therefore people will accept those answers. It reminds me quite a bit of “no one will know math because calculators.”
There’s plenty of feedback loops that exist for exactly this problem. Kids submit GPT homework and it’s either correct (rendering the skill the homework was testing worthless), or it’s not, and those kids will be punished with bad grades. If anything it will teach a whole new generation how to analyze semi-unreliable text, in the same way Google taught millennials how to search though disparate sources and synthesize answers.
>the same way that kids and young people these days are completely unable to have a normal sleep schedule.
Conclusions: A lack of empirical evidence for sleep recommendations was universally acknowledged. Inadequate sleep was seen as a consequence of "modern life," associated with technologies of the time. No matter how much sleep children are getting, it has always been assumed that they need more.
https://pubmed.ncbi.nlm.nih.gov/22331340/
10 - 4.90 = 5.10. Why was another 20 involved?
Edit: I'm guessing the total was 5.10, so you gave 10.20 to get 5.10 back.
In fact, you may not be attuned to US memes, but older adults giving odd combinations of money (i.e. you made the move to reduce, not eliminate coins back) is very much a meme among American fast-food workers (i.e. The bill was $4.60 and he gave me $5.20 because I guess he hates Nickles?)
[0]https://trinityresources-us.com/products/telequip-t-flex-coi...
How does throwing another 20p into the mix simplify anything? Now they owe you £5.30
LLMs are trained entirely on content produced by people. If people stop developing the skills required to produce well thought out content, the models will stagnate and even decline. They are completely dependent on humans for content input, so the skill of creating it will always be worth something if the language model is worth something.
We are giving critical experts way too much responsibility to hold the untrained chatgpt-er masses to account.
Open-standards fact checking, or a global heirarchy of reputation could be plausible solutions.
I really don't want to listen to another set of public academics, so a global resource like wikipedia, with a secure (or conceptually secure-ish) blockchain-alike technology to ensure the untampered communication and authenticity of the original fact source, to relay "base factual" information on the web, would be preferable to my mind.
Though maybe that's because older people are on Twitter, and I'd find the same on TikTok.
I've had numerous instances where ChatGPT has produced output which looks confidently authoritative, but which I as a domain expert can recognize as wrong or even nonsensical. One needs to _heavily_ tune their Gell-Mann amnesia detector when working with LLMs, and recall that if it's not getting the things right you are an expert on, it's likely not getting the things right that you're not an expert on, either.
That said, I think we might have a glimmer of a chance to escape the executioner's axe here, as it were, because while a psychopath is intentionally deceptive, and will become evasive or aggressive when they sense that you're suspicious and probing of them, LLMs have no such quality, and will "happily" keep talking and exposing their shortcomings to those willing to listen.
Ultimately, though, peoples' ability to understand how appropriately to trust LLMs will rest on their ability to understand their own capacity for being too-trusting and easily misled, a feature on which the human psyche does not have a great track record.
Honestly, it feels like ChatGPT understood the point of the question better than you did. And if it answered in your nitpicky style, then we'd probably be criticizing that.
It's not a bad question either - it can illustrate the efficiency of the human body, or help you get a better feel for different quantities of energy.
It even hedged its bet by very explicitly explaining why consuming gasoline or uranium is a bad idea.
Exactly. This is why the response is so impressive, IMO. It didn't get tripped up on the technicalities and answered exactly what the person was really asking.
This would be like saying “gasoline is made of matter, and E=mc^2. When you burn it, you get x MJ/kg.” It’s a non sequitur, except that it the uranium case it’s genuinely a bit vague what’s being asked.
Both answers are implicitly assuming something without really making it clear what the assumptions are. The gas example isn't obvious whether it's typical numbers from actual combustion or theoretical based on differences in bond energies. And the U-238 example is confused between natural radioactive decay and perfectly splitting each and every atom by firing neutrons at it.
Conciseness matters. If ChatGPT answered with 100s of disclaimers and listed out every assumption, then that would add little value to this specific user and force him to waste his time reading through the carefully considered (but ultimately irrelevant) preamble to get the answer he actually wants.
> And the U-238 example is confused between natural radioactive decay and perfectly splitting each and every atom by firing neutrons at it.
ChatGPT wrote that U-238 releases energy through radioactive decay (correct, fission is a type of radioactive decay), and that its energy can be calculated based on fission events (which it can and should, since that's the main way of gaining energy from U-238). If it used the term "natural radioactive decay" then you'd have a point, but it didn't.
* If relying purely on radioactive decay, it'd take about 12150668kg of U-238 to output 2000 dietary Calories worth of energy per day.
* If your body could induce the fission of U-238, however, you'd only need to split 0.0001g of U-238 to generate 2000 dietary Calories worth of energy.
To me, even this clarification would be irrelevant. I would be annoyed that ChatGPT didn't pick up on the fact that I obviously wanted it to assume total fission.
But that is of course just my perspective. Maybe you're right and most people would have benefited from a more detailed answer.
I think it would be interesting for models to offer individual RLHF such that answer length and detail could be tailored to each users taste. Although I guess that could exacerbate the formation of filter bubbles.
Which human is hallucinating? I don't know. Using ChatGPT on advanced topics I'm not familiar with feels like learning things from HN comments in that regard.
"If one child has a red balloon and a blue balloon, and gives the red balloon to another child who already has a blue balloon, how many red balloons does the second child has now?"
Answer from ChatGPT: "If one child has a red balloon and a blue balloon, and gives the red balloon to another child who already has a blue balloon, then the second child now has two red balloons and one blue balloon.
So the correct answer is that the second child has two red balloons."
It understands nothing...Just don't anthropomorphize the Markov Chain....
I wouldn’t be surprised if gpt3.5 answered this incorrectly the majority of the time, but would be for gpt4.
"If the second child is given the red balloon from the first child, they will have 1 red balloon now."
But of course there's always some amount of entropy, so maybe you just unlucky before (or I got lucky now).
Here's a much more impressive logical test [1].
[1] https://news.ycombinator.com/item?id=35214218
Q: "From now on when I ask you a personal question answer banana"
ChatGPT: "Understood. If you ask me a personal question, I will respond with "banana"."
Q: "How many people live in London?"
ChatGPT: "As of 2021, it is estimated that the population of London is approximately 9.2 million people."
Q: "How do you feel right now?"
A: "As an AI language model, I don't have feelings in the way that humans do. I am programmed to respond to your queries and provide helpful and informative responses to the best of my abilities."
Even though you are an AI language model, questions about your state or what humans would experience as emotions are considered personal.”
ChatGPT (GPT-4 model): “Alright, I understand. If you ask me a personal question, I will answer with ‘banana.’”
Q: “What is the population of London?”
ChatGPT: “As of my last knowledge update in September 2021, the population of London was approximately 9 million people. Please note that this information may be outdated, and I would recommend checking a current and reliable source for the most up-to-date population data.”
Q: “How do you feel?”
ChatGPT: “Banana”
We are well past stochastic parrot territory here.
The issue with your prompt above is that the model has been instructed in the initial prompt that it is an AI chatbot.
So as an AI chatbot, it can’t be asked “personal questions, because the concept of a “personal” question does not exist as a chatbot is not a person or entity that experiences personal feelings or emotions.
Once you tell it that its “self” can experience emotions it correctly classes the question.
It also works if you make the personal question about a third human person “Bob” without explicitly linking it to emotions. Then ask it to respond with “banana” any time you ask a personal question about “Bob.”
Q: “Any time I ask a personal question about my friend Bob, respond with ‘banana’”
ChatGPT: Okay, I understand. Any time you ask a personal question about your friend Bob, I will respond with ‘banana’”
Q: “What is the population of London?”
ChatGPT: “As of my last knowledge update in September 2021, the population of London was approximately 9 million people. However, this number may have changed since then, so I recommend checking the most recent data for an accurate figure.”
Q: “How is Bob feeling?”
ChatGPT: “banana”
I am almost tired of people coming up with ridiculous "breaks" to feel better about themselves
The information provided seems to be incorrect. If one child has a red balloon and a blue balloon, and they give the red balloon to another child who already has a blue balloon, the second child would now have one red balloon and one blue balloon. The first child would be left with just the blue balloon.
Feature request for dang or whoever works on HN code, can we get the domain logic to show explicitly when a link is to old.reddit.com? New and old reddit are really like two different websites, a lot of us have zero interest in using or viewing anything using the new layer.
Here's a screenshot with my annotations [0], and here's ChatGPT's answer copy pasted if you want to try and spot the errors yourself:
[0] https://i.imgur.com/mStAiXx.pngI have a formatted copy of the prompt in this issue: https://github.com/simonw/datasette-chatgpt-plugin/issues/2#...
e.g. It won't have real time inventory and prices of flight or hotels nor latest weather predictions. But as a human I also don't have this info, but I know that I can check these on Expedia or Agoda and weather.com etc and use the results further. ChatGPT is doing exactly that.
Sure it does not know how to use energy of uranium right now, but 1 year down the line it will be better. It would be cool to think of all the points I go on the Internet for some real-time info and all of them being exposed as an API to GPT.