The "Turing test" bar has risen quite high. This is IMHO already a huge problem when you are a moderator with limited resources, trying to fight a hostile attacker (wait until they start semi-randomly picking "write in style X" !), and especially without causing a lot of collateral damage (false positives).
This is what happens with lack of "dogfooding", as you can see, the lead developer hasn't been active there since ~2021 (and in 2019 posted "I haven't had time to look at the forum for many years anyway") :
Also my suspicion as to why the Path of Exile forums are so... inadequate (especially search), (though at least not overrun by spambots, in fact they might have the opposite issue with over-zealous moderation) you can hardly find the devs (including moderators) interacting with the community there these days :
Already getting this. Absurdist summaries of my company's history page that read like a 8th grade book report. Thing is I bet they use AI to parse responses, which means there's many opportunities for prompt-injection.
It seems you're mentioning Hacker News (HN) and suggesting there's an issue related to a profile there. How can I help you with this? Are you looking for advice on managing your HN profile, or is there a specific problem you're facing that you'd like assistance with?
Dang personally wrote me an email telling me to stop claiming to be an LLM, or rather, asking if I was and telling me to stop if so.
Back then I saw it as some kind of private art project to write controversial comments that looked like LLM output. Discussions under my comments regularly devolved into fights over whether I am one. Some would call it trolling I guess. I got a kick out of it. To be honest, I would love to continue!
But just to be clear: All the drivel I ever commented on here was organically produced by my average, contrarian and unhinged brain.
I guess part of the thinking I wanted to provoke was about the fact that nowadays you just can not know anymore if you are talking to an LLM or not and how the perception of whether you are changes how you communicate. About how your communication changes, based on a single boolean of metainformation (isHuman/isAI/isLlm).
Your comment is a bit worrying, since precisely zero humans I know in real life write like that, especially on Github issues, and the first sentence was enough to trip my mental "AI detector".
Yeah, that really off-putting mixture of slavish subservience, patronizing familiarity, and appeasing corpo-speak should give it away immediately to anyone who has read their share of LLM output.
It doesn't help that customer support humans have been trained to behave like that before chatbots became common. LLMs take it to the next level, but there's been a few times where I found out the entity on the other end of the conversation was actually a human, but the responses were entirely unhuman and unhelpful.
Well, it is called dosubot, it has a "bot" flair, the message starts with a robot emoji, and its writing style is very recognizably that of a "helpful AI assistant". Plus it links to https://dosu.dev/ at the bottom.
I don't know this project and I don't understand what this is. Are they using AI to generate package info, and then using a bot to handle their Github issues? What does the project actually do, that something so useless was deployed?
> What is the point of dosubot? It agrees, and apologizes for not having provided a supposedly adequate response.
This is my problem with chat gpt - I can’t have anything resembling a ‘real’ conversation with the bot, because it refuses to engage with me in any way other than:
1) restate my last message
2) agree with what I’ve said to an excessive degree, possibly even complimenting me on having said it
3) ask me a leading question that encourages me to send another message, and now goto 1 and loop forever.
It ends up just feeling like I’m the target of some kind of time wasting prank.
I think the problem is that AI is wrong so often that it would be foolish for it to stick to its guns when you're correcting it.
Bing Chat did this at first, and started calling the user a manipulator and an abuser when you corrected it too much (regardless of whether the previous message made any sense or not). I found it really funny, but other people were distressed by it because I suppose they thought they were talking to something sentient.
The problem is probably specific to the default system prompt in the default ChatGPT UI version.
An open weights LLM can be made into a pretty engaging bot with the right prompt/model/finetuning. I use an uncensored LLM with a special system prompt which makes it answer directly without all the fluff.
The one thing AI has done really well is generate all the signals of "intelligence" that SV tech folks care about. It's verbose and meaningless, confidently-asserted bullshit wrapped in the right packaging. It's like a Markov chain generator with a Stanford degree.
That's not a great argument for it being impressive, though, which is the original remark in this thread.
A car that unpredictably explodes and kills all its occupants, but which can be made safe by installing a widget, is not impressive (or at least, not in a good way).
It's verbose because it was finetuned to be "helpful" (the way OpenAI sees it). You can fix it with a system prompt, or finetune the base model with the format you want. Same with grounding it with RAG. Sure if you take a vanilla LLM and make no effort to adapt it to your app's needs, it's going to have subpar results. At least, verbosity is not an inherent problem of LLMs, it's a specific issue of a specific finetune. Hallucinations are a real problem indeed. However, being wrong and being very confident about your answer is something LLMs share with humans. LLMs can already have value if they're wrong less often than the average human (and some benchmarks suggest so).
This is part of a larger pattern I observe where people become fixated on solutions and forget (or never even knew) about the problem.
If you're working on something it's worth asking yourself every day: what is the problem I'm working on? Does this problem actually exist? Am I sure I'm working on a problem, or am I working on a solution?
> This is part of a larger pattern I observe where people become fixated on solutions and forget (or never even knew) about the problem.
This is just the natural behavior of trying to ride the wave of any tech hype cycle. There are incentives (both monetary and status) for doing so. FOMO also plays a large part here.
They know very well what they're doing and the tradeoffs of what they're doing. It's not a mistake, bias or hyperfixation.
> what is the problem I'm working on? Does this problem actually exist?
This is a nice guys finish last situation. The investor capital goes towards "solutionizing", so that's what people chase. Nobody in their right mind thinks food delivery or e-scooter rental is a billion dollar idea for example, but money flows into it anyway.
Wait until you realize your government already used this to run some minor aspect of your life and they're looking for ways to make more people redundant.
in my company they keep talking about ai ai ai and I have to keep explaining to them that it is NOT approved by the FDA to let some bug-stupid bit of vectors calculate critical data.
This is just the same lazy and dishonest people justifying their decisions as “efficient”.
Shitty, misinformative GPT-written block text. Cheap, Chinese-made, low-QC products on Temu or Amazon. Spammy, low-information SEO-optimized results at the top of Google search. Barely functional code written by an offshore team several corporate entities removed from the end users of that code. Shell subsidiaries to assemble Boeing planes to shift the responsibility of QC from the actual manufacturer.
It’s all the different angles of the same underbelly. Our favorite flavor of capitalism rewards the lowest denominator drivel, and perpetually validates that quantity over quality is a profitable and inevitably attractive trade off at sufficient scale.
The pkgx[.]dev website has some kind of `curl | sh` type installation commands to install packages. Given the AI generated descriptions of packages on their websites, I hope this isn't something other than an elaborate way to deliver malware to anyone who stumbles upon this website via a Google search.
I would highly encourage everyone that I strongly dislike to find random things with nonsensical artificial intelligence generated descriptions and then sudo curl | sh
> Hey @pawamoy! Great to see you back here, diving into the intriguing world of pkg descriptions. Let's take a closer look together!
> [and so on for 94 pages]
Nope, y'know what, close issue, I don't care any more. (If I were pawamoy.)
This kind of crap's even worse than when you get a 'stale' tag for inactivity. Sure, true, nobody else has said anything, but why does that discount my buggy experience or whatever? I would understand if it was because I didn't reply for so long after a maintainer response, but when it's just because there hasn't been any update it's just annoying. Nobody likes '+1's, so don't force even more notifications from label added, pointless comment added to bump, label removed!
> This kind of crap's even worse than when you get a 'stale' tag for inactivity. Sure, true, nobody else has said anything, but why does that discount my buggy experience or whatever? I would understand if it was because I didn't reply for so long after a maintainer response, but when it's just because there hasn't been any update it's just annoying. Nobody likes '+1's, so don't force even more notifications from label added, pointless comment added to bump, label removed!
I absolutely hate repositories that do this.
You're sending a clear signal that you don't care about bug reports from users. I bumped one issue for about a year (I seem to remember it was security-related too, to make matters worse) whilst the bot repeatedly tried to close it. It never got fixed.
Top tier comedy. Even the Github bot replies to the question are LLM nonsense: they simply agree with the above comment and apologize for not knowing the answer.
What's interesting is that this was created by Max Howell, creator of Homebrew who made waves a little while back for getting rejected by Google despite Homebrew's success. He talks about that here: https://www.quora.com/Whats-the-logic-behind-Google-rejectin...
To his credit, he didn’t let being rejected by the largest advertising surveillance company stop him from shipping surveillance software to millions of end-user workstations anyway, in the form of the nonconsensual spyware built into Homebrew.
It’s not anonymous, it includes the client IP address as well as a permanent supercookie unique identifier. Coupled along with client IP geolocation, this leaks your travel history to the server. It does so silently when you install packages, just like any other spyware.
The problem is an ethical one: just because they made software does not entitle them to the information about what does or does not happen on a machine that they do not own without the informed consent of the owner or operator of that machine. Assuming consent and opting the user in automatically is the issue.
Anything other than advance, informed consent is just spying.
Debian has figured it out. Why do other open source projects have such a hard time understanding consent?
We also always requested for Google to never store the IP address and now we run our own analytics infrastructure we definitely do not store it at all.
sneak just has a grudge against Homebrew because we blocked them for going on and on about this so they resort to ranting on Hacker News instead every time Homebrew is mentioned.
You are confused, despite my having explained many times. Your misrepresentation seems like bad faith. The Upton Sinclair quote comes to mind.
My issue is with all people who ship spyware; it is unethical. It’s nothing specific to Homebrew (sadly) and it’s not a grudge.
Lots of people would turn it off if they realized it was spying on them. The problem is people who assume consent and co-opt computers that do not belong to them to spy on users and exfiltrate data that isn’t theirs. They think that being volunteers or making f/oss entitles them to act unethically.
Opt-in consent is fine. These authors don’t use it because they secretly know that if they asked outright, most people would say no.
Seriously, it’s not any personal grudge against your project (and didn’t know until this moment that you had blocked me): Mattermost, VSCode, NetData, CapRover, and a thousand others also behave in such an unethical manner. It’s shameful, and it should be illegal.
Many people don’t know about it (as evidenced by sibling comments in this thread) and obviously should. This is a failure of the project that spreading public awareness about helps.
I would love to see the f/oss community abandon projects that are run in such an unethical manner. Nixpkgs on macOS, for example, doesn’t spy on you. (Neither do the package managers in the Linuxen you can replace macOS with.)
You're hijacking the term "ethical" to present yourself as morally superior. Many people don't think they're behaving unethically, and I think you're of the minority opinion.
> Many people don’t know about it and obviously should.
If people did know, what would be the consequence? What tangible difference would it make in anyone's life?
I just think it’s hard to appreciate how much work it takes to build quality software. And automated reporting really helps!
It sounds like Homebrew has its own reality distortion field, to be able to produce such pure unadulterated self-congratulatory BS like that.
As someone who had the unfortunate experience of having to try it once and interact with the "community", I am not surprised to see that it's still a bubble of delusion lead by someone trying to emulate Apple's cult.
I'm a fan of lihaoyi's response to similar tracking accusations.
> It's on by default. If you don't like that, turn it off. Or stop using Ammonite if you want to make a philosophical statement. I don't make any money off any of you, so I won't be particularly sad to see anyone go.
They don’t ask before, it happens without consent. The problem is claiming it’s anonymous when it’s not. Loading hacker news is also not anonymous. Calling something anonymous when it transmits your IP address is factually incorrect.
83 comments
[ 0.21 ms ] story [ 151 ms ] threadBehold, RimWorld's official forums : https://ludeon.com/forums/index.php?board=1.0
But yeah, it does seem to be getting worse as neural networks get better... :(
https://ludeon.com/forums/index.php?topic=49934.0
Also my suspicion as to why the Path of Exile forums are so... inadequate (especially search), (though at least not overrun by spambots, in fact they might have the opposite issue with over-zealous moderation) you can hardly find the devs (including moderators) interacting with the community there these days :
https://www.pathofexile.com/forum
A counter-example would be the Factorio forums... so far ?
https://forums.factorio.com/search.php?search_id=active_topi...
In the past we didn't have the resources to fill github with endless pages of blogspam, now we do.
Well done. It seemed like a very natural conversation.
Back then I saw it as some kind of private art project to write controversial comments that looked like LLM output. Discussions under my comments regularly devolved into fights over whether I am one. Some would call it trolling I guess. I got a kick out of it. To be honest, I would love to continue!
But just to be clear: All the drivel I ever commented on here was organically produced by my average, contrarian and unhinged brain.
I guess part of the thinking I wanted to provoke was about the fact that nowadays you just can not know anymore if you are talking to an LLM or not and how the perception of whether you are changes how you communicate. About how your communication changes, based on a single boolean of metainformation (isHuman/isAI/isLlm).
This is why chatbots suck for customer support.
Describes a lot of tech hype stuff in the last couple of decades.
This is my problem with chat gpt - I can’t have anything resembling a ‘real’ conversation with the bot, because it refuses to engage with me in any way other than:
1) restate my last message 2) agree with what I’ve said to an excessive degree, possibly even complimenting me on having said it 3) ask me a leading question that encourages me to send another message, and now goto 1 and loop forever.
It ends up just feeling like I’m the target of some kind of time wasting prank.
Bing Chat did this at first, and started calling the user a manipulator and an abuser when you corrected it too much (regardless of whether the previous message made any sense or not). I found it really funny, but other people were distressed by it because I suppose they thought they were talking to something sentient.
An open weights LLM can be made into a pretty engaging bot with the right prompt/model/finetuning. I use an uncensored LLM with a special system prompt which makes it answer directly without all the fluff.
A car that unpredictably explodes and kills all its occupants, but which can be made safe by installing a widget, is not impressive (or at least, not in a good way).
For example, 100 is defined as the average IQ, but it's almost certain that every human who comments on HN has an IQ much above that.
It's also problematic when LLMs are better not because they're getting better, but the average human is declining.
It went from "hey that's cool" to "wait people are actually relying on this" very quickly
This is part of a larger pattern I observe where people become fixated on solutions and forget (or never even knew) about the problem.
If you're working on something it's worth asking yourself every day: what is the problem I'm working on? Does this problem actually exist? Am I sure I'm working on a problem, or am I working on a solution?
This is just the natural behavior of trying to ride the wave of any tech hype cycle. There are incentives (both monetary and status) for doing so. FOMO also plays a large part here.
They know very well what they're doing and the tradeoffs of what they're doing. It's not a mistake, bias or hyperfixation.
This is a nice guys finish last situation. The investor capital goes towards "solutionizing", so that's what people chase. Nobody in their right mind thinks food delivery or e-scooter rental is a billion dollar idea for example, but money flows into it anyway.
Shitty, misinformative GPT-written block text. Cheap, Chinese-made, low-QC products on Temu or Amazon. Spammy, low-information SEO-optimized results at the top of Google search. Barely functional code written by an offshore team several corporate entities removed from the end users of that code. Shell subsidiaries to assemble Boeing planes to shift the responsibility of QC from the actual manufacturer.
It’s all the different angles of the same underbelly. Our favorite flavor of capitalism rewards the lowest denominator drivel, and perpetually validates that quantity over quality is a profitable and inevitably attractive trade off at sufficient scale.
> Hey @pawamoy! Great to see you back here, diving into the intriguing world of pkg descriptions. Let's take a closer look together!
> [and so on for 94 pages]
Nope, y'know what, close issue, I don't care any more. (If I were pawamoy.)
This kind of crap's even worse than when you get a 'stale' tag for inactivity. Sure, true, nobody else has said anything, but why does that discount my buggy experience or whatever? I would understand if it was because I didn't reply for so long after a maintainer response, but when it's just because there hasn't been any update it's just annoying. Nobody likes '+1's, so don't force even more notifications from label added, pointless comment added to bump, label removed!
I absolutely hate repositories that do this.
You're sending a clear signal that you don't care about bug reports from users. I bumped one issue for about a year (I seem to remember it was security-related too, to make matters worse) whilst the bot repeatedly tried to close it. It never got fixed.
They’re collecting errors and statistics to make the product better. It’s all anonymous and you can turn it off.
If it were Facebook ads, I’d be more sympathetic. But it’s automated bug reporting…
The problem is an ethical one: just because they made software does not entitle them to the information about what does or does not happen on a machine that they do not own without the informed consent of the owner or operator of that machine. Assuming consent and opting the user in automatically is the issue.
Anything other than advance, informed consent is just spying.
Debian has figured it out. Why do other open source projects have such a hard time understanding consent?
We also always requested for Google to never store the IP address and now we run our own analytics infrastructure we definitely do not store it at all.
sneak just has a grudge against Homebrew because we blocked them for going on and on about this so they resort to ranting on Hacker News instead every time Homebrew is mentioned.
My issue is with all people who ship spyware; it is unethical. It’s nothing specific to Homebrew (sadly) and it’s not a grudge.
Lots of people would turn it off if they realized it was spying on them. The problem is people who assume consent and co-opt computers that do not belong to them to spy on users and exfiltrate data that isn’t theirs. They think that being volunteers or making f/oss entitles them to act unethically.
Opt-in consent is fine. These authors don’t use it because they secretly know that if they asked outright, most people would say no.
Seriously, it’s not any personal grudge against your project (and didn’t know until this moment that you had blocked me): Mattermost, VSCode, NetData, CapRover, and a thousand others also behave in such an unethical manner. It’s shameful, and it should be illegal.
Many people don’t know about it (as evidenced by sibling comments in this thread) and obviously should. This is a failure of the project that spreading public awareness about helps.
I would love to see the f/oss community abandon projects that are run in such an unethical manner. Nixpkgs on macOS, for example, doesn’t spy on you. (Neither do the package managers in the Linuxen you can replace macOS with.)
> Many people don’t know about it and obviously should.
If people did know, what would be the consequence? What tangible difference would it make in anyone's life?
And you’re sending your IP address all over the internet. Is Hacker News violating your privacy because they have a log an incoming requests?
It’s not like Homebrew is identifying YOU by name or cross referencing the UUID on your machine to advertise to you…
I just think it’s hard to appreciate how much work it takes to build quality software. And automated reporting really helps!
Living in a “trustless” world is a myth.
It sounds like Homebrew has its own reality distortion field, to be able to produce such pure unadulterated self-congratulatory BS like that.
As someone who had the unfortunate experience of having to try it once and interact with the "community", I am not surprised to see that it's still a bubble of delusion lead by someone trying to emulate Apple's cult.
> It's on by default. If you don't like that, turn it off. Or stop using Ammonite if you want to make a philosophical statement. I don't make any money off any of you, so I won't be particularly sad to see anyone go.
https://www.reddit.com/r/scala/comments/6irnix/about_ammonit...