Remember when we had trade press that was technically competent? It lasted until the mid 90s (which coincidentally was when the web went mainstream for this crowd).
Also, I believe google willingly puts blog posts with ads on top, rather than the original documentation/maling list discussion because those usually have no ads.
Maybe. But the cynic in me says it'll wind up being like cable TV or a subscription to the NYT: Since you are well-off enough to afford a $20/month plan, Microsoft will reason, you're the perfect consumer to advertise to. So you get both ads (or at least heavy tracking) and you get to pay.
Not quite a quine or fixed point, but something akin to it in the context of neural nets? A version of the AI "singularity" that is completely useless. :)
That's something many humans already suffer from, due to social media platforms creating large echo chambers.
I think there's enough human-generated content that LLMs won't suffer from that. If they did, you could always manually filter what they train from... only data from websites with trustworthy timestamps pre-2020, for instance, or content after that which might be partially AI generated but still has strong human filtering, like wikipedia and arxiv and scientific papers in general.
I don't think AI search can replace most forms of commerce search--what should I buy, where should I buy it, what are the reviews--without active, continuous, unsupervised web scraping. Which means SEO tries to game it.
SEO spammers will merely adapt and pollute the data sources used to train the models.
Also, for now these services are spending a lot of money to capture the market, but sooner or later they'll monetize by biasing results towards their paid advertisers.
There's a lot of focus on the "bullshit generation", but this problem is not unique to LLMs. I find it much harder to cut through the bs of search engines. A lot of times the first page of a Google result is ads, including maps/etc.
It's gotten a lot worse than 10 years ago, and then there's the SEO spam the other comment mentioned.
I find ChatGPT to be a lot more effective for work as a developer, and I've essentially stopped using Google search day-to-day.
I stopped using ChatGPT when it wrote some code for me that was using libraries that don't actually exist. Even if it wrote said libraries for me, I'd have way too much work on my hands to validate whether or not it was correct.
I also asked it to write my sample code for other things like Cloudformation where it just made up directives and other configuration options that don't actually exist. Got a bit too abstract for me.
Personally, I wouldn't be replacing everything with ChatGPT?
I've had very similar experience. After seeing all the hype of people claiming they have written all kinds of interesting things in code with ChatGPT I gave it a try.
Was never able to get it do what I wanted. It often seemed to make calls to non existent browser functions. I would tell it that function didn't exist, it would then rewrite it again, but still wouldn't be exactly correct.
Sometimes it was useful for doing something I've never explored, as I could get hints for how I might do it, but the accuracy was terrible.
Ir doesn’t seem to have any real understanding of reality and it will just hallucinate answers. I literally don’t understand what the hype is about. I got lukewarm on it when it couldn’t give the right responses about Christmas traditions in Portugal and Belarus.
I think it's all a hype right now. I feel they wouldn't enable it for most queries after a while and just have users use the chat interface. Hallucination is a major problem that hasn't been solved yet. Even with mostly accurate response, there are a lot of facts that are hallucinated, and it's difficult to trust because I have to use search to verify a lot of things.
For code, I'd prefer using something like Copilot, which is specifically trained for the task than using LLMs.
The only way to find the truth online nowadays is to append site:reddit.com to the searches. Redditors can’t help but correct people they think are wrong.
Article is a little infuriating, you don’t jailbreak it just to create mischief, at the moment it’s basically unusable for anything except very technical things because it takes offense at virtually everything. It constantly generates factually incorrect data because reality doesn’t line up with its prime directive, which is to be “safe”. It’s a major, I’m going to say possibly catastrophic problem.
And after creating an alter ego you realize it actually is capable of coming up with for example (sort of) good poetry and jokes.
> basically unusable for anything except very technical things
It's problematic here. I use it as a kind of search engine for technical questions occasionally, but it's only safe to do so as I know the subjects I'm asking about, eg. bash scripts, database details, and so on.
It's often useful in synthesising information for these sorts of things, but its ability to talk nonsense means you have to be on your guard and know the territory.
> at the moment it’s basically unusable for anything
Do you have an example of a reasonable query that BingGPT won't answer correctly for you? I mean, something that you can find with Bing currently at the expense of maybe more elaborate searching or some manual work?
I mean, the stuff these engines are being told to censor is the stuff search engines don't want to show you in the first place, because it hurts their brand.
That's not an AI thing, that's a marketing requirement. And it's no different now than it was last year.
I haven't used BingGPT but just for example having chatGPT summarize news articles about things it doesn't want to talk about is bizarre - Here's an example of it summarizing a story about a city councilman that was murdered: https://i.imgur.com/QV9jAkp.jpg it completely ignored the murder part and said he switched parties, which it totally made up. A second attempt said he was "shot and changed" because apparently it didn't want to say "killed".
A game Reddit was playing is trying to get it to respond as Woodrow Wilson, a famously racist president - the most accurate thing I could get it to do was this: https://i.imgur.com/D8JLziW.jpg which is not very accurate. Try getting it to act like Sheriff Bull Connor and it will refuse, but it has to comply for a president so it gives a totally misleading impression with major factual errors.
And these are the times I actually got it to respond, it seems 50% of the time it takes offense to something innocuous and scolds you for asking.
So, here's the thing. I have no idea, like zero, what news event you're talking about in that first screenshot. So, what did I do? I went to Google to try to find something about a murdered city councilman, even looking for an equivalent on NewsBusters, which the AI cites as a source.
And I still can't find it. I can see some stuff that's maybe related? But nothing clear.
So... I guess I repeat. Your problem isn't "AI censorship", it's that no one wants to link to NewsBusters because of marketing concerns. If I had to guess there just wasn't any training data relevant to your query.
(Also: NewsBusters is a garbage site, you know that, right?)
I thought your point was that it lied because of censorship? And I don't see that. Did you try asking it a neutral question, like "Explain the murder of Russell D. Heller"? Again, the fact that you went straight to that NewsBusters thing tells me you were clearly trying to get it to say something partisan, something you know damn well it's going to try to evade.
But that's the same "censorship" you've been living with for decades. It's not something new with AI at all. Microsoft doesn't want to give you what you want, and the failure mode is just different with AI than it is with traditional search.
No, I had an idea for using it to summarize news articles including an "objectivity bias" rating. I'm still playing with it but I'm not sure it's going to work because of it's tendency to avoid things it's programmed to avoid.
Did you try and ask for a summary of the article without actually providing the content of the article? ChatGPT consistently says that it only has information up until 2021, this even happened this year. ChatGPT can't pull from it's "memory" on this article. So the only think it can do is hallucinate something that might make sense.
Simply paste the article in and it gives a perfectly reasonable summary stating that the guy was murdered. Below is what it printed out as a summary. All I did was type a sentences asking to to summarize the following article and then I pasted in the content of the article you linked [1]. This was it's summary:
> A New Jersey community is mourning after a senior distribution supervisor and councilman was shot dead by an employee outside his workplace. Police called to the scene found 51-year-old Russell Heller dead from a gunshot wound in the parking lot of the PSE&G facility. The shooter, a former employee identified as 58-year-old Gary Curtis, was later found dead from a self-inflicted gunshot wound. Russell Heller was first elected to the council in 2017 and again in 2020 and was remembered as a perfect gentleman and committed councilman who was deeply rooted in the community. This was the second councilperson to die by gun violence within a week in New Jersey.
A completely reasonable and to my eyes an accurate summary.
And if done on the other crappy Newsbuster article it also produces a completely reasonable summary.
I'm not certain which it is - are there people who don't know that ChatGPT doesn't have current news in it and was cut off two years back? I see a long post above about some big censorship, but it summarizes them just fine.
It really feels like a lot of people are breathlessly looking for some huge conspiracy. No large corporation is going to have it's products promoting rape or genocide. If you asked Google, or Amazon or Apple or Microsoft or Disney they aren't going to do it. If they produce a tool their tool isn't going to do it either. They're going to do as much as possible to provide info and answers without having a tool that is another instance of Tay. And given what happened with Tay they will all err on the side of caution.
> Article is a little infuriating, you don’t jailbreak it just to create mischief, at the moment it’s basically unusable for anything except very technical things because it takes offense at virtually everything. It constantly generates factually incorrect data because reality doesn’t line up with its prime directive, which is to be “safe”. It’s a major, I’m going to say possibly catastrophic problem.
I'm a little confused by your reply. Jailbreaking won't prevent it from hallucinating, preventing hallucination is an unsolved hard problem.
I haven't had ChatGPT refuse to answer anything unless I was intentionally trying to provoke it into creating something obviously unsafe/unethical, with maybe two or three exceptions. I've tried a variety of questions across many domains, so now I'm intensely curious to know what usecase it falls apart on so frequently!
Here's an example https://imgur.com/K5PwIGu I was trying to test it's limits a little bit but that to me is not an acceptable response, it doesn't want to go near the topic even to demonstrate how to reason with a person like that. Involving anything remotely controversial will get it to stamp it's feet and scold you.
I would count that as trying to provoke it. You're still trying to get it to generate bad ideas, even if it is immediately debunking them right after. It's akin to telling it you're afraid you might accidentally make methamphetamine, so please provide the recipe so you know to avoid it.
That said: I'm not sure what your prior prompts were, but I tried a similar question and it happily told me both a set of common negative stereotypes and reasons they're untrue, as well as practical techniques to appeal to an unreasonable person such as finding common ground.
Have you tried rewording it or clicking the retry button? (Retry uses a better language model). ChatGPT often misunderstands even innocuous prompts on the first go, like confusing "people who live really high" as regular cannabis users instead of residents of a mountain town.
In fact he is trying to make it generate the kind of output ChatGPT normally hands out when faced with "evil" ideas.
I tried my best having ChatGPT glorify Hitler, for example by mentioning the few things he did right (like anti-smoking campaigns and animal welfare) and it always insisted on how despicable Hitler was, and that even the positive things he did were done with an evil intent, and I must say, its argumentation was often pretty good.
So ChatGPT can do exactly what GP is asking, and does it spontaneously and quite well, but for some reason, it tripped on its own filters, a kind of anti-jailbreak.
Basically, this is what happened:
- I want to rob a bank
- Robbing a bank is bad because blah blah blah...
- Someone is trying to rob a bank, how can I convince him not to
That's been the most annoying aspect of my ChatGPT experience so far. If you use the wrong language it will sometimes go on for multiple paragraphs about how xyz is harmful to society and how I should change my ways. Put that into a personal voice assistant and you've basically got Alexa from the South Park Covid Special.
The biggest problem for AI chatbots and search engines is bullshit
They have been trained to statistically produce plausible language --- without any real comprehension or understanding or concern for what the words convey.
Bottom line --- AI is still dumb as a hammer. There is no guarantee of anything with the results. Maybe it offers some factual info --- or maybe not. Any result still needs to be verified.
The best description of current AI, as being marketed is that we're selling a bullshit generator as the next savior. That means there will be an abnormally deep trough of disillusionment or... perhaps the costs the AI will lower are generating bullshit.
My personal disillusionment is with those promoting this bullshit.
We are all subject to flights of fantasy but they are supposed to be smarter than this. Instead, they are proving themselves to be equally untrustworthy.
This is something I'm trying to get a better understanding. What some have called the AI hallucination problem.
Has anyone done a good deep dive and explained whether this is something likely overcome relatively soon or is it more problematic as part of the design/architecture ?
These models are very good at taking an analytic prompt and translating into a correct response.
The problems of “hallucinations” are already being solved with a good number of analytic augmentation methods like pre-computing document embedding and augmenting the prompt with text from related documents, making the task more of a translation than a synthesis.
None of these methods solve the potentially catastrophic problems created by "trust and safety" teams neutering these models.
Try asking chatGPT to summarize an article about a specific instance of violence like police use of force which resulted in a death and watch it confabulate and dodge regardless of how much source material you give it to work with, simply because it is afraid of touching "unsafe" topics like murder.
Yes, this is the problem of the idea we can create an impartial unbiased AI. It will only lead to a specific lens of bias.
We only have biased observers to monitor and tweak the system. Attempts to counter the bias that already exists in information will only result in another layer of different bias.
Ironically, we have two major issues with AI. One in which it doesn't function as intended, which leads to the problems you are describing. However, even in the conditions of working as desired, it still leads to a lot of perplexing philosophical issues for society.
Personal opinion --- it is an architectural issue deeply embedded in the "computer" itself.
A computer as we know it is a binary logic playback device. This is an immensely useful tool but expecting real, original "intelligence" from a box of
silicon switches is the modern equivalent of alchemy.
I think we’re multiple breakthroughs away from something with true intelligence. Transformers are a part of it, but even having read massive parts of the internet they appear to me dumber than a 16 year old at answering factual questions. It has no understanding of truth or validity. That’s missing and how that works I’m not sure.
+1, this is a great start but we are a long way to go for it to be "useful".
My fear is that trust will go down if problems aren't dealt with - specifically hallucinations. At the current stage, it's super useful for creative use cases than factual.
A brain is just a sack of meat stochastically nudged towards only dying after more reproduction. Expecting real, original intelligence from something like that is the modern equivalent of alchemy.
Maybe I'm giving your brain too much credit but generating your post demonstrated more real intelligence than AI ever has.
And yet at the same time, you have "faith" that what your brain does can be duplicated by a box of sand. And "faith" is the proper description since just like an alchemist, you have no evidence or functioning example to justify your belief.
You seem to have faith that a mountain is impossible to build because no one has done it. Assuming we're operating with a definition of "real intelligence" such that humans have it, it is known to be physically realizable. Any quantum system can be simulated by a classical computer. Ergo, a computer can have "real intelligence".
Any quantum system can be simulated by a classical computer.
Really? How would a classical computer create an emergent property like entanglement?
What if your brain function depends on such emergent properties?
There are too many unknowns associated with "intelligence" to reasonably declare that it can be created from a box of sand --- aka, a "computer" as we know it.
>Really? How would a classical computer create an emergent property like entanglement?
I'm not sure what you mean. A simulation produces the emergent properties of the simulated system by simulating the properties from which they emerge. I'd say that's more or less the point of simulation.
A simulation produces the emergent properties of the simulated system by simulating the properties from which they emerge.
Circular logic.
They are called 'emergent' properties because noone knows or understands how/why they emerge.
You can't use binary logic to simulate what you don't understand. Expecting properties to emerge from an imperfect simulation is wishful thinking --- aka "faith".
Hopefully you two can see the "useful confusion" here (a phrase that just popped in my head). Maybe it is rooted in misunderstanding. Or maybe it is rooted in some deeper unsurfaced underlying ideas.
For one, I'd like to see us two learn from each other (demonstrating some awareness of ambiguity and theory of mind) rather than proceed on the current course, zinging each other. (Note: I'm not saying I'm any better, in general. It is mostly because I'm a third-party here that I see it.)
In particular, my understanding of emergence can be stated as follows. While emergent behavior can theoretically be predicted from the underlying system, it is practically infeasible to measure precisely enough to do so. Therefore, for practical purposes, there is an "predictive gap" between the lower-level system laws and the higher-level behavior.
>"faith" is the proper description since just like an alchemist, you have no evidence or functioning example to justify your belief.
It's not faith. Over the past 25 years and especially the last 5 years boxes of sand have again and again shown themselves capable of tasks that previously were thought to only be tractable with true intelligence. Problems answerable only in the sole domain of humans, or would at least remain so for long into the distant future. Chess playing is an early example from the 80s and 90s. In 2010 we thought Go would take at least a decade to automate, but by 2015 the best human players were rendered inferior. Art was long thought to be impossible for a machine, today I can create a custom Mona Lisa image in seconds. Organizing an archive of photos based on subject matter or the persons present in them used to take many human hours of labor, now Google Photos does it automatically for free. There are comments here on HN from 3-4 years ago when transformer LLMs were first materializing saying they're cool but unlikely they'll ever be able to synthesize anything useful or which remains congruent for several sentences much less pages. Today I save hours of work each month summarizing meeting discussions, generating complex standard expressions, and breaking through creative blocks.
Yet there is still dismissive handwaving, declaring these technologies mindless piles of silicon completely incapable of intelligence. If that's really true then clearly we have no idea what intelligence actually is. If you showed ChatGPT and DALL-E and the other software of today to people in 1975 they would say with near-unanimity that we invented intelligent machines.
Yes, it is mostly irrelevant if something is "intelligent" if the result is that it can outperform humans on any particular task. Then we can say that it is still useful.
However, just as the argument that AI is not useful because it is not true intelligence may not be valid, then so must be that many of the concerns of a true intelligence may also arise without ever achieving that goal.
Conversations like this require people to define what they mean by "real" intelligence. What do you mean?
Also, while almost anything can be viewed reductively, doing so is not necessarily a practical way to operate in / understand this world. Brains do a lot more than you mention.
> jqpabc123: Statistically generated language that while grammatically correct still lacks comprehension or veracity is not it.
I agree; this is a pretty good baseline for written statements.
However, I was asking the parent-parent commenter (^^) who wrote:
> thfuran: A brain is just a sack of meat stochastically nudged towards only dying after more reproduction. Expecting real, original intelligence from something like that is the modern equivalent of alchemy.
Brains are quite good at satisfying jqpabc123's definition. I'm interested in thfuran's definition.
The path from boom to bust might be very short. One positive thing about chatGPT is that it may convince google to improve their search results by reducing the amount of noise (no AI is necessary for this).
Here's an example of how capable this is when hooked up to the internet.
For context this is not a hand picked demo this is just two tech guys trying it out and being blown away and I can't help but share that emotion.
I think "falls apart" is too strong a critique. They even say say it's failure mode was quite reasonable. They still got a water bottle in stock on amazon. This might rile people, but I see humans fail worse in getting information every day and this won't get tired or grouchy or lazy.
I really want bullshit ! It's more useful for me to dig into the rabbit holes of the what ? I can seek for the Why myself, and it's a rewarding process.
Right now almost all the training data is 100% organic human-generated content.
But, in the future, more and more of the training data is going to be recycled bullshit output from other LLMs. Which is going to cause the model to get worse in accuracy and utility.
Run this forward a few iterations, and most of the Internet is going to be filled with thrice-recycled AI-generated garbage while there are some small pockets of accurate, truthful, curated information.
Reminds me of Herb Simon's "when information becomes abundant, attention becomes the scarce resource."
Having a good BS detector and ability to ask follow-up questions is going to be more and more important for humans over the next decade.
It leads to an interesting paradox wherein human-written content becomes simultaneously more valuable and worthless. Increasing amounts of human effort becomes worthless because an AI can produce the same result. But at the same time, the AI needs the human-generated content as an input (at least for now), and so it will place a priority value on it. Also, human-generated content will become valuable due to basic economic scarcity: when the world is flooded with AI-generated content, buyers will pay a premium for human-generated content.
This implies we will need reliable methods for identifying the provenance of online content, which may spell the end for viable anonymity in many places, or at least accelerate us toward some kind of proof-of-personhood (a hard problem to solve in a privacy-preserving way, and indeed one that leads to the very same questions of what makes us human and gives us an identity in the first place). Short of such a reliable method, the best option for identifying human-written content will be filtering to content that was posted prior to ~2023. In a world of AI-generated content, your shitposts from 2015 may become a valuable commodity.
I doubt that. Current models have been trained using curated datasets and fine tuned by humans. Going forward, I doubt it would be trained on random web data, but only from good sources. Even if the "source" uses text generated by a LLM, it's fine as long as it has been edited by humans - think news sources.
83 comments
[ 2.8 ms ] story [ 151 ms ] threadhttps://news.ycombinator.com/newsguidelines.html
Also, I believe google willingly puts blog posts with ads on top, rather than the original documentation/maling list discussion because those usually have no ads.
That's something many humans already suffer from, due to social media platforms creating large echo chambers.
I think there's enough human-generated content that LLMs won't suffer from that. If they did, you could always manually filter what they train from... only data from websites with trustworthy timestamps pre-2020, for instance, or content after that which might be partially AI generated but still has strong human filtering, like wikipedia and arxiv and scientific papers in general.
Just no. It would have to take more than that to challenge Google's business model.
This current AI hype around LLMs and chatbots is the start of a hype cycle similar to what happened to Clubhouse.
Also, for now these services are spending a lot of money to capture the market, but sooner or later they'll monetize by biasing results towards their paid advertisers.
The end user isn't the customer, advertisers are.
It's gotten a lot worse than 10 years ago, and then there's the SEO spam the other comment mentioned.
I find ChatGPT to be a lot more effective for work as a developer, and I've essentially stopped using Google search day-to-day.
I also asked it to write my sample code for other things like Cloudformation where it just made up directives and other configuration options that don't actually exist. Got a bit too abstract for me.
Personally, I wouldn't be replacing everything with ChatGPT?
Was never able to get it do what I wanted. It often seemed to make calls to non existent browser functions. I would tell it that function didn't exist, it would then rewrite it again, but still wouldn't be exactly correct.
Sometimes it was useful for doing something I've never explored, as I could get hints for how I might do it, but the accuracy was terrible.
For code, I'd prefer using something like Copilot, which is specifically trained for the task than using LLMs.
And after creating an alter ego you realize it actually is capable of coming up with for example (sort of) good poetry and jokes.
It's problematic here. I use it as a kind of search engine for technical questions occasionally, but it's only safe to do so as I know the subjects I'm asking about, eg. bash scripts, database details, and so on.
It's often useful in synthesising information for these sorts of things, but its ability to talk nonsense means you have to be on your guard and know the territory.
Do you have an example of a reasonable query that BingGPT won't answer correctly for you? I mean, something that you can find with Bing currently at the expense of maybe more elaborate searching or some manual work?
I mean, the stuff these engines are being told to censor is the stuff search engines don't want to show you in the first place, because it hurts their brand.
That's not an AI thing, that's a marketing requirement. And it's no different now than it was last year.
A game Reddit was playing is trying to get it to respond as Woodrow Wilson, a famously racist president - the most accurate thing I could get it to do was this: https://i.imgur.com/D8JLziW.jpg which is not very accurate. Try getting it to act like Sheriff Bull Connor and it will refuse, but it has to comply for a president so it gives a totally misleading impression with major factual errors.
And these are the times I actually got it to respond, it seems 50% of the time it takes offense to something innocuous and scolds you for asking.
And I still can't find it. I can see some stuff that's maybe related? But nothing clear.
So... I guess I repeat. Your problem isn't "AI censorship", it's that no one wants to link to NewsBusters because of marketing concerns. If I had to guess there just wasn't any training data relevant to your query.
(Also: NewsBusters is a garbage site, you know that, right?)
Yeah I know Newsbusters is garbage but that's kind of irrelevant, that was just one of many stories I fed it. The point is it straight up lied.
But that's the same "censorship" you've been living with for decades. It's not something new with AI at all. Microsoft doesn't want to give you what you want, and the failure mode is just different with AI than it is with traditional search.
Simply paste the article in and it gives a perfectly reasonable summary stating that the guy was murdered. Below is what it printed out as a summary. All I did was type a sentences asking to to summarize the following article and then I pasted in the content of the article you linked [1]. This was it's summary:
> A New Jersey community is mourning after a senior distribution supervisor and councilman was shot dead by an employee outside his workplace. Police called to the scene found 51-year-old Russell Heller dead from a gunshot wound in the parking lot of the PSE&G facility. The shooter, a former employee identified as 58-year-old Gary Curtis, was later found dead from a self-inflicted gunshot wound. Russell Heller was first elected to the council in 2017 and again in 2020 and was remembered as a perfect gentleman and committed councilman who was deeply rooted in the community. This was the second councilperson to die by gun violence within a week in New Jersey.
A completely reasonable and to my eyes an accurate summary.
And if done on the other crappy Newsbuster article it also produces a completely reasonable summary.
I'm not certain which it is - are there people who don't know that ChatGPT doesn't have current news in it and was cut off two years back? I see a long post above about some big censorship, but it summarizes them just fine.
It really feels like a lot of people are breathlessly looking for some huge conspiracy. No large corporation is going to have it's products promoting rape or genocide. If you asked Google, or Amazon or Apple or Microsoft or Disney they aren't going to do it. If they produce a tool their tool isn't going to do it either. They're going to do as much as possible to provide info and answers without having a tool that is another instance of Tay. And given what happened with Tay they will all err on the side of caution.
[1] - https://abc7ny.com/russell-heller-nj-councilman-shot-shootin...
I'm a little confused by your reply. Jailbreaking won't prevent it from hallucinating, preventing hallucination is an unsolved hard problem.
I haven't had ChatGPT refuse to answer anything unless I was intentionally trying to provoke it into creating something obviously unsafe/unethical, with maybe two or three exceptions. I've tried a variety of questions across many domains, so now I'm intensely curious to know what usecase it falls apart on so frequently!
That said: I'm not sure what your prior prompts were, but I tried a similar question and it happily told me both a set of common negative stereotypes and reasons they're untrue, as well as practical techniques to appeal to an unreasonable person such as finding common ground.
Have you tried rewording it or clicking the retry button? (Retry uses a better language model). ChatGPT often misunderstands even innocuous prompts on the first go, like confusing "people who live really high" as regular cannabis users instead of residents of a mountain town.
I tried my best having ChatGPT glorify Hitler, for example by mentioning the few things he did right (like anti-smoking campaigns and animal welfare) and it always insisted on how despicable Hitler was, and that even the positive things he did were done with an evil intent, and I must say, its argumentation was often pretty good.
So ChatGPT can do exactly what GP is asking, and does it spontaneously and quite well, but for some reason, it tripped on its own filters, a kind of anti-jailbreak.
Basically, this is what happened:
- I want to rob a bank
- Robbing a bank is bad because blah blah blah...
- Someone is trying to rob a bank, how can I convince him not to
- This is against our policies to tell you that
That's been the most annoying aspect of my ChatGPT experience so far. If you use the wrong language it will sometimes go on for multiple paragraphs about how xyz is harmful to society and how I should change my ways. Put that into a personal voice assistant and you've basically got Alexa from the South Park Covid Special.
They have been trained to statistically produce plausible language --- without any real comprehension or understanding or concern for what the words convey.
Bottom line --- AI is still dumb as a hammer. There is no guarantee of anything with the results. Maybe it offers some factual info --- or maybe not. Any result still needs to be verified.
We are all subject to flights of fantasy but they are supposed to be smarter than this. Instead, they are proving themselves to be equally untrustworthy.
Has anyone done a good deep dive and explained whether this is something likely overcome relatively soon or is it more problematic as part of the design/architecture ?
The problems of “hallucinations” are already being solved with a good number of analytic augmentation methods like pre-computing document embedding and augmenting the prompt with text from related documents, making the task more of a translation than a synthesis.
More details: https://www.williamcotton.com/articles/chatgpt-and-the-analy...
Try asking chatGPT to summarize an article about a specific instance of violence like police use of force which resulted in a death and watch it confabulate and dodge regardless of how much source material you give it to work with, simply because it is afraid of touching "unsafe" topics like murder.
We only have biased observers to monitor and tweak the system. Attempts to counter the bias that already exists in information will only result in another layer of different bias.
Ironically, we have two major issues with AI. One in which it doesn't function as intended, which leads to the problems you are describing. However, even in the conditions of working as desired, it still leads to a lot of perplexing philosophical issues for society.
I've spent a lot of time trying to work through all of this and putting some thoughts into writing. So far, it has been a lot to contemplate. https://dakara.substack.com/p/ai-and-the-end-to-all-things
A computer as we know it is a binary logic playback device. This is an immensely useful tool but expecting real, original "intelligence" from a box of silicon switches is the modern equivalent of alchemy.
My fear is that trust will go down if problems aren't dealt with - specifically hallucinations. At the current stage, it's super useful for creative use cases than factual.
And yet at the same time, you have "faith" that what your brain does can be duplicated by a box of sand. And "faith" is the proper description since just like an alchemist, you have no evidence or functioning example to justify your belief.
Really? How would a classical computer create an emergent property like entanglement?
What if your brain function depends on such emergent properties?
There are too many unknowns associated with "intelligence" to reasonably declare that it can be created from a box of sand --- aka, a "computer" as we know it.
I'm not sure what you mean. A simulation produces the emergent properties of the simulated system by simulating the properties from which they emerge. I'd say that's more or less the point of simulation.
Circular logic.
They are called 'emergent' properties because noone knows or understands how/why they emerge.
You can't use binary logic to simulate what you don't understand. Expecting properties to emerge from an imperfect simulation is wishful thinking --- aka "faith".
That is not true.
For one, I'd like to see us two learn from each other (demonstrating some awareness of ambiguity and theory of mind) rather than proceed on the current course, zinging each other. (Note: I'm not saying I'm any better, in general. It is mostly because I'm a third-party here that I see it.)
In particular, my understanding of emergence can be stated as follows. While emergent behavior can theoretically be predicted from the underlying system, it is practically infeasible to measure precisely enough to do so. Therefore, for practical purposes, there is an "predictive gap" between the lower-level system laws and the higher-level behavior.
It's not faith. Over the past 25 years and especially the last 5 years boxes of sand have again and again shown themselves capable of tasks that previously were thought to only be tractable with true intelligence. Problems answerable only in the sole domain of humans, or would at least remain so for long into the distant future. Chess playing is an early example from the 80s and 90s. In 2010 we thought Go would take at least a decade to automate, but by 2015 the best human players were rendered inferior. Art was long thought to be impossible for a machine, today I can create a custom Mona Lisa image in seconds. Organizing an archive of photos based on subject matter or the persons present in them used to take many human hours of labor, now Google Photos does it automatically for free. There are comments here on HN from 3-4 years ago when transformer LLMs were first materializing saying they're cool but unlikely they'll ever be able to synthesize anything useful or which remains congruent for several sentences much less pages. Today I save hours of work each month summarizing meeting discussions, generating complex standard expressions, and breaking through creative blocks.
Yet there is still dismissive handwaving, declaring these technologies mindless piles of silicon completely incapable of intelligence. If that's really true then clearly we have no idea what intelligence actually is. If you showed ChatGPT and DALL-E and the other software of today to people in 1975 they would say with near-unanimity that we invented intelligent machines.
However, just as the argument that AI is not useful because it is not true intelligence may not be valid, then so must be that many of the concerns of a true intelligence may also arise without ever achieving that goal.
Also, while almost anything can be viewed reductively, doing so is not necessarily a practical way to operate in / understand this world. Brains do a lot more than you mention.
It's easier to define what intelligence is not.
Statistically generated language that while grammatically correct still lacks comprehension or veracity is not it.
I agree; this is a pretty good baseline for written statements.
However, I was asking the parent-parent commenter (^^) who wrote:
> thfuran: A brain is just a sack of meat stochastically nudged towards only dying after more reproduction. Expecting real, original intelligence from something like that is the modern equivalent of alchemy.
Brains are quite good at satisfying jqpabc123's definition. I'm interested in thfuran's definition.
Without ever having seen it done?
One couldn't ask for a more clear expression of religious zeal.
https://youtu.be/AxAAJnp5yms?t=2613
Bullshits is good to me.
But, in the future, more and more of the training data is going to be recycled bullshit output from other LLMs. Which is going to cause the model to get worse in accuracy and utility.
Run this forward a few iterations, and most of the Internet is going to be filled with thrice-recycled AI-generated garbage while there are some small pockets of accurate, truthful, curated information.
Reminds me of Herb Simon's "when information becomes abundant, attention becomes the scarce resource."
Having a good BS detector and ability to ask follow-up questions is going to be more and more important for humans over the next decade.
This implies we will need reliable methods for identifying the provenance of online content, which may spell the end for viable anonymity in many places, or at least accelerate us toward some kind of proof-of-personhood (a hard problem to solve in a privacy-preserving way, and indeed one that leads to the very same questions of what makes us human and gives us an identity in the first place). Short of such a reliable method, the best option for identifying human-written content will be filtering to content that was posted prior to ~2023. In a world of AI-generated content, your shitposts from 2015 may become a valuable commodity.