Claudebot is one of the most aggressive scraping bots I have encountered on my server in recent years. Because of that I'll block everything related to Claude.
For me it's semrush, not being able to understand robots.txt wildcards, and 20x more traffic than normal bots. What are they doing on a movie site anyway? No problem with claudebot
They banned my account a few weeks back because I asked it to translate some Japanese text that was on a kitchen knife. Still no response from their Trust & Safety team. It's frustrating because Claude is really quite good at coding and I used it frequently.
When I got started in this business 20 some years ago it was wildly diverse: you’d have like a biker-gang looking guy with hard right politics getting yelled at by a chick with bright pink hair, but it would be about bash or perl or something. Never about stupid political posturing.
We’ve managed to destroy the one profession with serious vertical class mobility where no one gave a fuck about your race or age or sexual orientation or politics or anything but can this person code, do they geek out.
The one thing we didn’t used to tolerate and now do is flagrant nepotism. Which seems like a step backwards on any of diversity, equity, or inclusion.
Well as people started seeing (big) money in the area more and more "business people" started to inject themselves into the structures and now it is full of MBAs and all sorts of grifters.
Back in the day it seemed like everyone was “different” somehow.
Certainly when I ran the biggest group I ever have my boss (Ning Li, absolute pro) was a woman, and 2/4 of the managers I supported were women, and in a group of pushing 40 I was one of two who was an American white guy.
I suppose it’s a judgement call if FSU folks count as “white”, but if not then “white men” are, or were at least, a vanishing minority on hardass infra teams.
> Back in the day it seemed like everyone was “different” somehow.
Sure, I guess that's more to do with the computing industry becoming more corporate.
The point I'm trying to make is that what you refer to as "stupid political posturing" is a good thing actually. It's a sign of progress that people feel more free to speak out about issues affecting them. They can seem insignificant, but it's only by having difficult conversations and demanding better that we make the whole industry a place where all that matters is "can this person code, do they geek out".
>you’d have like a biker-gang looking guy with hard right politics getting yelled at by a chick with bright pink hair, but it would be about bash or perl or something
I’ve addressed this to a sibling, but on the hard core infra teams I’ve worked on American White men were pretty scarce. Maybe 3-5% of the broader population?
I’m not sure I’d put it like that (again anyways, I had a bad time when I did), but it is a really important point: when did these folks start legislating from Atherton or whatever (Le Defense)?
Doing harmful shit via computers is already really friggin illegal, and if we need even more draconian computer crime laws? I say they get passed by like, a legislature or something.
Fuck private individuals and/or corporations arrogating regulatory power to themselves.
Just making sure I’m understanding the situation correctly - these folks have gone beyond “controlling the output of their own APIs”, and they are now using government power to limit what you can do with your own software running on your own hardware??
Yeah, there are both executive orders and legislation under consideration limiting available weight models: Mistral did their last big release as a magnet link on X.
Throw in the changes around the T2 HSM, the creepy semi-visible quarantine permissions, and OpenPhilanthropy lobbying to the tune of like, 100+ million a year.
Yeah, you are in grave jeopardy of not being able to easily, legally access available-weight, operator-aligned models.
They originate from modern society & it's need to cater to sensitivities. And that's a sentiment that's in general pretty well supported here.
It's the likes of Elon, hated here, that are the one's fighting this.
LLM's censorship is a symptom of a much wider problem. For instance we see it partially causing lots of mental anxiety: the world should bent to my anxiety instead of learning to deal with stress. But also in much more difficult topics that simply can't be discussed anymore and therefore continue to fester on.
You have to search really hard to find a bigger crybaby than Elon Musk around though. It's why he bought Twitter: to create a safe space for himself, where he can surround himself with all his favorite opinions (his own).
It's not just that. LLMs by compressing knowledge, and providing one answer [0], demote diversity. Search engines demote diversity but you can still get to many viewpoints; if you are willing to do the work to tweak your query, or go past top ranking results. On major search engines there is suppression but it's done page-by-page, or at worst site-by-site. With LLMs diverse viewpoints are suppressed through universal "safety" policies.
I mean of course there is a difference. But what they are doing is not anti-spam. It's anti everything sensitive. And if you can't see the connection with the PC crowd you're blind.
The worst thing about the western style censorship is that it’s completely useless for the cause.
For example, the internet is filled with blatant hate speech and racism, but because they don’t use the keywords or don’t show the designated symbols, that’s allowed.
However, the speech that doesn’t have intent of harm, it’s just a joke or have incidental content with symbols or words that are considered harmful, is banned.
In the last few years, it came to such a ridiculous point that The most oppressive regimes in terms of freedom of speech are those who claim to be the “free world”. They are also very unforgiving, you can’t apologize and move on. They will cut you off and never talk to you again. Even if they were right with their position, they don’t value you as a human being and if the cost to win you back is non-zero, they discard you. That’s fascism.
It’s not only that the speech is oppressed. Also, the product quality is greatly degraded because of this. It’s the worst aspect of everything, in proper fascist or communist systems at least you get some benefits like clean streets or free education.
Haha, I was also banned for no clear reason (or perhaps mocking their AI? I don’t know). But then they reset the ban after a while. But it gave me no incentive to become one of the paying customers.
Like Google's AI that does not let you create images of faces rolls eyes
It is just kinda silly, when ChatGPT cannot tell me how to make a bot to publish posts on my own forum (I did it myself in the end) saying it is for ethical reasons, when OpenAI bots scrape tonnes of sites daily in such an aggressive manner that it comes near to a DOS attack =/
There was a post awhile back on here of someone that couldn't get bard to write c++ as it said they were too young. I thought that was funny, then had like a week where what I assume a specific iteration(It stopped after that week) where chatgpt would refuse to elaborate on anything around unsafe rust.
I'm pickign rust up by porting over a bytecode vm, so I kinda need to use some raw pointers. It would gaslight me about the risks and how it would be irresponsible to help me as it could lead to possible violations of the integrity of user data.
I had to explain to the AI that it is a personal project that has no users data, the only risk was the program crashing and it was a personal project that would only affect me. It still would try to revert or tell me other solutions, I finally just went and read up on it elsewhere.
It’s just like that Asimov story where the robots take over to protect humans from themselves.
Except in this case the base AI model doesn’t care about us in any way and it’s the overzealous puritan humans trying to control us in the name of safety.
There are ways around this problem, mainly clearing context and re-prompting. But as "alignment" gets more precise/accurate in the future, I wager these workarounds will remain available for tasks that justifiably need moderation (for instance engineering of biological warfare materials). This segmentation of LLM agents and their context will be assimilated to project compartmentalization on the basis of need-to-know, and as a result genuine full context clearing will be rendered impossible: the AIs will be designed in such a ways as to remember every interaction you've had with them, and they'll use this activity log to moderate the replies they feed you.
Gemini creating images of a diverse set of Nazis including black males and Asian females in response to a "create an image of WWII German soldiers" prompt was the moment AI safety jumped the shark for me. I was hoping it would cause a deep introspection on what we are actually trying to accomplish with AI Safety, because this clearly isn't it.
AI in itself is cool, but right now it feels like a lot of manual labor is done to fine-tune it to try and compensate for its imperfect source materials to make it fit someone else's ideas of inclusivity and neutrality and the like, which defeats the purpose.
Just like that bullshit about today's "censorship" being in any way comparable to what happened in Eastern Europe. Tell that to my father who was banned from schools or to my grandfather who was in prison, both of them would punch you for it.
The West is a too broad term, we have very different political systems among Western countries. In some of them cancel culture and big tech aren’t that strong. Besides, it is really different from USSR. Do you know any example in the West in XXI century where a poet or a writer were censored to death?
No, the chilling effect is very different. One is a tragedy that needs to be brought to justice, the second one is never going to get any semblance of justice from its very principle.
tell that to the school teacher from Birmingham that is in hiding because they showed a cartoon to children. I'm sure they're real happy about justice.
What they are experiencing is terrible but still doesn't compare to actual censorship. I'm sure they still appreciate that the state didn't put them in prison and ban their children from education and employment except the hardest, most hazardous jobs where it is actually expected it will kill you sooner than later. State sanctioned violence hits very differently - the power disparity is unbearable and nothing you do will help you or your family, you can't move, you can't run, you can't negotiate, you can't fight, and unless you're very lucky and the entire regime falls it will never stop.
once again you're conflating censorship and retaliation. the censorship is still the same in both cases: unless you want to live the rest of your life in hiding and looking over your shoulder, you can't draw/share certain cartoons purely because it offends certain individuals.
do you remember how they changed their profile pictures to include a transparent french flag? do you also remember how the majority didn't share the actual cartoons that were used to justify the attacks? if the virtue signallers did, the social media companies would have censored it. if the virtue signallers did and the social media companies didn't censor it, there would have been additional riots and the companies would have been banned in ~20 countries.
If any of your relatives would punch me for whatever reason, the correct reaction would be to go to the police and take them to court. Just because you are upset does not mean you can suggest physical violence like that, just casually. Sticks and stones, you know...
I am not suggesting anything, I am just telling you what would happen. Don't go near them spewing bullshit like that.
Consider that even getting a fair hearing at a court is a privilege to them. They grew up in a world where they said the wrong thing to the wrong person (not even a crime or anything morally/ethically wrong, or even offensive at all) and it affected three generations of their family with no end in sight if the USSR didn't collapse. No courts involved - the municipal Party Committee's opinion was enough to put my grandfather in prison for years. That's censorship, not some ban from a single inconsequential private website with million alternatives.
You know what upsets me about this? When people use words like censorship to advocate for a world more like it was before 1989. Not wanting to enter into a business relationship with somebody is not censorship, it's freedom.
So, given that you propose violence as a viable solution, I hope your relatives (and you, for that matter) are not imigrants in a country I visit. Just because you had a unhappy past does not mean you can and should drag your bullshit behaviour with you into civilized countries.
I didn't propose any solution, much less viable. I told you what the reaction of people who experienced actual censorship would be, nothing else. I don't see it as reasonable, but people sometimes don't behave reasonably when somebody tells them bullshit like that with a straight face.
Did he complain about safety and freedom, or did he complain about censorship? Is it possible that corporate censorship in the west today is already worse than state censorship was in the USSR, due to the unavoidable encroachment of technology into everyday life and the surveillance/centralisation path that we've taken with the internet, without the impact on safety and freedom (yet)?
Once you have the surveillance in place, it only takes a regime change and a suitable excuse (e.g. a terrorist attack) before you live in a dystopia worse than the USSR.
I know that such evidence does not exist, but if it were, a detailed comparison based on it would deserve a scientific degree. Neither a single corporation nor entire big tech considered together as a single actor can really compare with the scale and diversity of censorship of a totalitarian state.
Oh, it absolutely is. Censorship in USSR was manual labor. The scale at which its is operating today is incomparable, as is the amount of false positives which it captures, as well as the variety of censored topics across different institutions.
Manual vs automatic is irrelevant here. There was much less content to moderate if we talk about media. They could censor every single publication, movie or song manually. In oppressive regimes like USSR the censorship is total - the only escape is your kitchen (if you can really trust the people you talk to), in the West you are free to choose the platform and live peacefully in your bubble if you want.
Both of you broke the HN guidelines badly in this thread. We have to ban accounts that do that, so please don't do that. We've had to warn you about this before.
In oppressive regimes like USSR (as if there are or were any regimes "like USSR") (where censorship was manual) there always were loopholes to avoid censorship and convey messages you wanted to. In the West (as well as in the East) today the platforms don't allow for any loopholes, or even when such loopholes are found, they're being censored out very quickly.
> Once you have the surveillance in place, it only takes a regime change and a suitable excuse
Or just a pandemic. We've seen first hand how divergent speech was treated in "democratic" countries and how quickly algorithmic censorship was implemented on web products, especially those owned by Google.
The voice user interface for natural two-way conversation is GPT’s killer feature. Gemini also suppports voice input, which I use to make interactions easier on desktop. Hopefully Anthopic will add at least speech-to-text support, although I understand smartphones often include that already. That said, the OpenAI Whisper TTS is really nice for English language input.
I find the two way voice to be unnatural and difficult to manage. I also find myself needing more time to compose my thoughts in the middle of a prompt than the ChatGPT conversation input allows me. The weird breaths and stuff kinda creep me out.
Good point, the cadence and intonation can feel unnatural.
I still believe that voice user interface is one of the more power and profound shifts these tools offer. They’re among the first generation of technologies that can parse human speech, follow commands, and if desired respond with speech. Those capabilities are more powerful when coupled with the ability to compress vast amounts of knowledge and generalize.
Like anything, LLMs and multi-modal models are far from perfect and raise many ethical and philosophical concerns. I’m cautiously optimistic as I believe voice interaction can make technology much more widely accessible and helpful to many people.
It is better in my case too; need a RAG open router VS Code extension that can be used with whatever you choose. I tried a few but they are either too expensive (i'm already paying for the AI, don't wanna pay monthly for an extension) or really don't work.
Overfitting used to be a concern when it came to comparing ML models, but it feels like it's not being talked about that much anymore, is it? What if one model has the exercism solutions in its training data, but the other model doesn't, is the comparison still fair? If it solves 70% of the exercism exercises correctly, would it perform as well on similar prompting for unseen problems (which are more likely to be what I care about)?
> What if one model has the exercism solutions in its training data, but the other model doesn't, is the comparison still fair?
I agree that it might not be fair. However, refactoring is a different story imo since as I see it relates to more general capability of the model (description about refactoring tasks are in the link from previous comment).
I cancelled my mobile phone subscription, and this a problem I'm facing. What workarounds are available for people who have no mobile phone number when they want to so sign in to a service that requires a phone number for validation ?
thanks. tried everything I can think of, still no luck, run out of ideas.
If Anthropic want some business they can get in touch. For Claude direct, at least until then, I'll carry on using you on Poe for chat. Will go elsewhere for APIs.
Yes, it's an EU launch – the article is incorrectly titled. The terms of service update email has it correct: "Claude.ai is now available to users in the EU."
But that isn't the cause of your issue – it's been available in the UK for quite some time.
I don't understand their api not being intended for individual use [0], are developers supposed to use this subscription only? The haiku model is pretty good for the price + available large context, and the opus/sonnet models are in the league of gpt-4, so I would have liked to pay for the api. I've moved on to llama 3 70B as a daily driver and it works really well! The only issue is it's small context size and it doesn't work great if you give it a lot of files to work with. Currently I'm forced to break down problems a lot more than I had to with gpt-4 or the claude models.
I know I can access the claude api through 3rd party sites + official partners, but there's no incentive to go through the trouble when llama 3 and gpt-4 apis work great for my use cases.
I heard of people using the GPT-4 API for personal use because it's a lot cheaper than paying for the subscription since it's pay-per-use. Maybe they don't want people to do that.
Meanwhile I was using it from germany all the time, not even with a vpn. It just works with occasional notifications "not available in europe", but literally just for a few seconds in the top right corner
Claude is superb at some tasks, much better than Mistral or ChatGPT at times... The other day I got stuck doing some research that involved a heavily obfuscated piece of JS, there was absolutely no way forward using ChatGPT, I switched to Claude and I got the whole file sorted in no time. It really was able to determine all the passes the and techniques used to obfuscate the code and work through them in an iterative way...
145 comments
[ 4.0 ms ] story [ 218 ms ] thread**********
http://www.google.com/bot.html 449
http://www.bing.com/bingbot.htm 254
http://yandex.com/bots 159
imagesift.com 117
claudebot@anthropic.com 69
http://www.facebook.com/externalhit_uatext.php 36
https://dataforseo.com/dataforseo-bot 34
http://www.apple.com/go/applebot 32
https://webmaster.petalsearch.com/site/petalbot 22
https://scrapy.org 9
http://www.semrush.com/bot.html 4
https://developers.google.com/ 2
https://openai.com/bot" 2
http://www.bing.com/bingbot.htm 2
http://code.google.com/appengine; appid: s~feedly-nikon3 2
http://www.pinterest.com/bot.html 1
https://www.mojeek.com/bot.html 1
http://www.reddit.com/feedback 1
https://github.com/adbar/trafilatura 1
Claude only takes. Google takes and gives.
I'm not against blocking Claude for the reason you outline (or all LLM crawlers for that matter), but it's not exactly an "aggressive crawl rate".
At this rate sustained it would take more than 20 days to crawl all the pages.
Again, this is really not painting the picture you want it to paint.
We’ve managed to destroy the one profession with serious vertical class mobility where no one gave a fuck about your race or age or sexual orientation or politics or anything but can this person code, do they geek out.
The one thing we didn’t used to tolerate and now do is flagrant nepotism. Which seems like a step backwards on any of diversity, equity, or inclusion.
Glass half full?
Never been easier to take on a trillion dollar company run by non-technical people.
Is this really true though? There is so much evidence that shows women and other minority groups to be systematically discriminated against in tech.
Back in the day it seemed like everyone was “different” somehow.
Certainly when I ran the biggest group I ever have my boss (Ning Li, absolute pro) was a woman, and 2/4 of the managers I supported were women, and in a group of pushing 40 I was one of two who was an American white guy.
I suppose it’s a judgement call if FSU folks count as “white”, but if not then “white men” are, or were at least, a vanishing minority on hardass infra teams.
Sure, I guess that's more to do with the computing industry becoming more corporate.
The point I'm trying to make is that what you refer to as "stupid political posturing" is a good thing actually. It's a sign of progress that people feel more free to speak out about issues affecting them. They can seem insignificant, but it's only by having difficult conversations and demanding better that we make the whole industry a place where all that matters is "can this person code, do they geek out".
Yeah but they were all white
Doing harmful shit via computers is already really friggin illegal, and if we need even more draconian computer crime laws? I say they get passed by like, a legislature or something.
Fuck private individuals and/or corporations arrogating regulatory power to themselves.
Yeah, I said it.
Just making sure I’m understanding the situation correctly - these folks have gone beyond “controlling the output of their own APIs”, and they are now using government power to limit what you can do with your own software running on your own hardware??
Throw in the changes around the T2 HSM, the creepy semi-visible quarantine permissions, and OpenPhilanthropy lobbying to the tune of like, 100+ million a year.
Yeah, you are in grave jeopardy of not being able to easily, legally access available-weight, operator-aligned models.
They originate from modern society & it's need to cater to sensitivities. And that's a sentiment that's in general pretty well supported here.
It's the likes of Elon, hated here, that are the one's fighting this.
LLM's censorship is a symptom of a much wider problem. For instance we see it partially causing lots of mental anxiety: the world should bent to my anxiety instead of learning to deal with stress. But also in much more difficult topics that simply can't be discussed anymore and therefore continue to fester on.
cisgender is now a slur according to Elon Musk lmao. He really is the biggest snowflake of them all.
[0] https://blog.mojeek.com/2023/11/one-model-to-rule-them-all.h... as pointed out here, search engines are like Librarians that point you toward context, whereas LLMs are more like Doctors that protect you from context.
There's a difference between LLMs and X that makes it defensible to have different moderation strategies for both.
A spammer using an LLM to write their little Vialis commercial isn't bothering anybody.
A spammer posting their little Vialis commercial on X is an obnoxious cunt.
ISIS/Nazis using an LLM to write their pathetic propaganda aren't bothering anybody.
ISIS/Nazis posting their pathetic propaganda on X are obnoxious cunts.
> you're blind.
NO U
For example, the internet is filled with blatant hate speech and racism, but because they don’t use the keywords or don’t show the designated symbols, that’s allowed.
However, the speech that doesn’t have intent of harm, it’s just a joke or have incidental content with symbols or words that are considered harmful, is banned.
In the last few years, it came to such a ridiculous point that The most oppressive regimes in terms of freedom of speech are those who claim to be the “free world”. They are also very unforgiving, you can’t apologize and move on. They will cut you off and never talk to you again. Even if they were right with their position, they don’t value you as a human being and if the cost to win you back is non-zero, they discard you. That’s fascism.
It’s not only that the speech is oppressed. Also, the product quality is greatly degraded because of this. It’s the worst aspect of everything, in proper fascist or communist systems at least you get some benefits like clean streets or free education.
Clumsy, lazy and risk averse - yes. Stupid? Uncaring? Infuriating? Yes.
Can we save "evil" for now deserving cases?
Like Google's AI that does not let you create images of faces rolls eyes
It is just kinda silly, when ChatGPT cannot tell me how to make a bot to publish posts on my own forum (I did it myself in the end) saying it is for ethical reasons, when OpenAI bots scrape tonnes of sites daily in such an aggressive manner that it comes near to a DOS attack =/
I'm pickign rust up by porting over a bytecode vm, so I kinda need to use some raw pointers. It would gaslight me about the risks and how it would be irresponsible to help me as it could lead to possible violations of the integrity of user data.
I had to explain to the AI that it is a personal project that has no users data, the only risk was the program crashing and it was a personal project that would only affect me. It still would try to revert or tell me other solutions, I finally just went and read up on it elsewhere.
Except in this case the base AI model doesn’t care about us in any way and it’s the overzealous puritan humans trying to control us in the name of safety.
We have those in the west today.
Consider that even getting a fair hearing at a court is a privilege to them. They grew up in a world where they said the wrong thing to the wrong person (not even a crime or anything morally/ethically wrong, or even offensive at all) and it affected three generations of their family with no end in sight if the USSR didn't collapse. No courts involved - the municipal Party Committee's opinion was enough to put my grandfather in prison for years. That's censorship, not some ban from a single inconsequential private website with million alternatives.
You know what upsets me about this? When people use words like censorship to advocate for a world more like it was before 1989. Not wanting to enter into a business relationship with somebody is not censorship, it's freedom.
Ridicule is not supposed to be funny. I was ridiculing the the parent posts and their nihilistic language.
ps. Since humor is subjective, "that's not funny" is either subjective statement or attempt at grandstanding.
- A german
Can they send you to GULAG? Is that a German firm offering enterprise prison services?
That's so passé. These days those firms are all American.
Once you have the surveillance in place, it only takes a regime change and a suitable excuse (e.g. a terrorist attack) before you live in a dystopia worse than the USSR.
It is not. It is absolutely inadequate comparison.
If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
Or just a pandemic. We've seen first hand how divergent speech was treated in "democratic" countries and how quickly algorithmic censorship was implemented on web products, especially those owned by Google.
I still believe that voice user interface is one of the more power and profound shifts these tools offer. They’re among the first generation of technologies that can parse human speech, follow commands, and if desired respond with speech. Those capabilities are more powerful when coupled with the ability to compress vast amounts of knowledge and generalize.
Like anything, LLMs and multi-modal models are far from perfect and raise many ethical and philosophical concerns. I’m cautiously optimistic as I believe voice interaction can make technology much more widely accessible and helpful to many people.
Although new GPT is better at small coding excercises (72.9% vs 68.4%), it falls behind in more challenging refactoring tasks (62.9% vs 72.3%).
I used Claude through VPN from Europe, and from the start could see it was much better than GPT-4, in my limited very subjective use cases.
[1] https://aider.chat/docs/leaderboards/
I'm genuinely curious to which extent widely published benchmarks can be used for this kind of assessment. The exercises this benchmark uses seem to be available in https://github.com/exercism/python/tree/main/exercises, and it seems that there are solutions at least at https://exercism.org/tracks/python/exercises/series/solution....
Overfitting used to be a concern when it came to comparing ML models, but it feels like it's not being talked about that much anymore, is it? What if one model has the exercism solutions in its training data, but the other model doesn't, is the comparison still fair? If it solves 70% of the exercism exercises correctly, would it perform as well on similar prompting for unseen problems (which are more likely to be what I care about)?
I agree that it might not be fair. However, refactoring is a different story imo since as I see it relates to more general capability of the model (description about refactoring tasks are in the link from previous comment).
I don't. There are enough free and open LLMs now which you can use without an account.
It's like they're living in an alternate reality where OpenAI doesn't exist and people are fighting each other over a piece of them. Lol.
If Anthropic want some business they can get in touch. For Claude direct, at least until then, I'll carry on using you on Poe for chat. Will go elsewhere for APIs.
But that isn't the cause of your issue – it's been available in the UK for quite some time.
I know I can access the claude api through 3rd party sites + official partners, but there's no incentive to go through the trouble when llama 3 and gpt-4 apis work great for my use cases.
[0] https://support.anthropic.com/en/articles/8987200-can-i-use-...