I was thinking, we should all contribute to a Yandex fund for the Ukrainian mil to help keep the level of bot activity on the rest of the internet under control!
Of course they have bots... That's how you index the web.
If we took Google offline I bet there would also be a reduction in bot traffic from them too!
Indexing is also probably a low priority task, so if half their datacenter capacity is taken offline, their bot traffic probably reduces by more than 50%.
Yandex has cloud offering, their servers are used for actual bot traffic, not just crawlers. Yandex hasn’t been only a search company for a very long time.
I've reported high-volume exploit scanners multiple times, and when expiring blocks a week later, they typically would show up & resumt with the same IP at some point.
Maybe they'd care if the account was using a stolen credit card and there was a chance they wouldn't get their money, but they definitely do not care otherwise for normal people like me. I'm sure that'd be different if it's a multinational or a country-level government body is reporting something, but I can't get those to do my bidding.
Right, because I've had such outstanding results getting in touch with actual humans from Google when reporting abusive traffic directed towards my IP ranges. Top tier attention from people who really care.
The downvotes are because you're resisting easily-observed reality with the seemingly-rational response "Where is this 'air'? If I'm breathing it, it should be easy to point to, no?"
No. If you can't see it, and don't understand it's there, you cannot be explained to. You're someone with no palate asking how anyone else can know if a wine tastes good.
I like to study rhetoric, and appreciate folks asking for specifics. There's a lot of generalities thrown around on this forum that are highly questionable, and the only antidote I have is to request details. It's surprising how often folks resist.
A sibling comment replying to you is using a familiar tactic, which is to try to shame you for asking for specifics, while simultaneously providing none and telling you that it's obvious.
Occasionally people do provide details and I learn something, but that's not the common case, sadly.
> Can you point to a few comments that you consider examples?
Just search for “Russia” on HN and follow threads. You’ll find lots of “what about West/Jews”, “Russia is fighting against NATO”, etc. Then open any western news outlet on YouTube and follow comments too.
Logic says google bots do not come disguised, but as google agents.
Or is there other information?
Now there is probably also AI agent spam from google, but not at this level I believe. Also I am not fully clear what they count as spam, I doubt they would include google in this list, so do they include yandex bots here, if they come officially?
There's also the very well known inverse phenomenon of absolutely non-google people trying to disguise their crawler as a googlebot useragent, which has been a thing since at least... 15 years now?
It's a nice reminder that HN is just filled with the same primitive tribal normies and is always getting oneshotted by the same transparent xenophobic hysteria that plagues the rest of the internet.
I keep thinking about them often, what a definition of cancer on modern humanity that country is. Not all people inside, far from it, but that country - absolutely, definitely and consistently.
And most of those inside allowed this to happen over decades since even harsh dictatorships represent underlying population in many ways, many even cheered / still cheer on direction their country has taken, and non-trivial amount also like raiding neighbors for the kill, loot and money in general.
A stark warning to US if you keep letting things slide to evil direction, this is roughly how you end up over time.
What an interesting way to phrase things when the US is responsible for much more suffering and unrest outside of it's own borders. Including the current situation in Russia, come to think of it.
I always wondered why every single western country is subjected to a constant stream of the same simplistic and generic anti immigration comments, regardless of the country. It's as if suddenly everyone from Poland to Portugal had the same moron racist uncle flooding their social media service of choice. There is also and always a russian-backed populist far-right emergent political party posting PG-13 versions of the same posts pushed by these bots.
I wonder what kind of fluctuation we will see in these bots traffic.
Its a deliberate attack being done by actors such as Elon musk. How you say? They are making the world think bots and AI automation are ok by supplying basic tools to help everybody scrape while they use technology 100x far more advanced to make everybody else's scraping pointless. They are making all Americans fight with each other and then using their huge data centers to harvest all of the data and keep users on their social media platforms. They basically use a fake Twitter account to "retweet" everybody's news or information and take it off their website into the social realm. They are delibertly sending more and more bots into the world so that they aren't noticed as much. If they didnt.. everybody would be pissed at their AI slop. People like ai meta have far advanced residential ips setup almost undetectable while all the new scraper people are easily detected but their ego and greed will not let them stop.. They are being trained to think it's ok to bot another person and steal their stuff because it's "seo" and "not illegal" "if I don't. Somebody else will" the bad actors here are Elon musk and Meta AI. There is no doubt about it. Don't be fooled by this fraud. Their setups to run their bots is easily 100000 a month... And nobody will be able to compete at that scale and expertise without just getting blocked.
They say it's 60% percent but that's not even close to the truth. In my niche the bots are now about 100 to 1. Yes. For every 1 visitor I get it's 100 automated bots using different ips.. after tracing most of them with fingerprint analysis.. the main culprits are Facebook and Elon musk. Meta is using something called "muse ai" to feed all the data from these bots and Elon musk appears to be channeling everything to x. I think that's what the X logo stands for.. the stocks and market.. everybody going down while he goes up.. the X is their plan on the economy and I just told you for fact how they are doing it.. yet all the morons who have been doing Internet work for 3 years are incapable of understanding. They just hire an Indian worker to fuel the fire
The HTTP traffic from Yandex is down to ~ 0 % after the third strike, after the first two strikes already degraded it. I wonder how this looks like at the network level because their search page still serves, but presumably their crawler is inactive now
Edit: Here is some more info, they do have availability issues as is to be expected
> Because three of its primary facilities were knocked offline within four days, Yandex’s failover mechanisms broke down[1][8].
> Outages have cascaded across Yandex Go (taxis), Yandex Pay, food delivery, smart home platforms, internal logistics, and third-party corporate services hosted on Yandex Cloud[2][7]. Problems have spilled into neighboring countries using these systems (Belarus, Kazakhstan, Armenia, Serbia)[2][7].
The KZ region wasn't intended for Russia's entire workload, that's a secondary region with better Central Asian access.
If what you deduct were correct, Yandex would not be telling its customers to switch to competitora like VK Cloud. They even have a "priority onboarding code" called "CloudBro" for three competitors.
Their latest consumer plan includes "Super Duper Bot Fight Mode", which lets you drone strike one data center in a sanctioned country of your choosing.
For additional strikes or to upgrade to ICBM-based payloads, you'll have to contact them to set up an enterprise plan.
I'm going to be really interested how much this cleans up the internet, I mean the damage is done if you read comment sections on youtube etc, but let's see how long the BS persists.
I think this really highlights the changing face of war.
It's hard to go to war now when your enemy can just use a few small drones to take out everything your citizens care about in a single day (social media, online deliveries etc).
And these things are practically irreplaceable in a war when you're sanctioned up the wazoo. Where are you buying racks of Xeons and DDR5 RAM and Nvidia GPUs?
Absence of bot traffic is secondary to the absence of nearly all Yandex traffic in general, bot or not, as a result of the datacenter being down. Which is certainly newsworthy.
Your initial submission title sounded as if Yandex DC was used as a bot attack and because they were down, bot traffic in the whole internet was substantially down.
Don't worry, now that Trump started protecting the refineries, the other data centers will be hit sooner rather than later.
All those fresh fp2 drones can't go to waste.
The only question is whats next - power plants? Water treatments? Heating somewhere up north?
Everyone is mentioning crawlers, but wouldn't most traffic from a datacenter's ASN usually be bots just in general given that there's not many humans there to generate human traffic?
Here's an example French hosting company which has something like 14.8% "human" traffic, (the Yandex is under 5% human) though how much of that is actual humans and not advanced things puppeteering headless browsers is another question.
I can’t believe there are so many dumb comments in this thread.
All it shows is that crawling from Yandex search engine stopped from one Yandex data center ASN.
No one runs opinion bots from data center IPs that people mention here.
On the other hand there are plenty of datacenter located command/control systems that puppeteer vast fleets of residential proxies (and Russia is by no means the only one doing so), so it will be very telling to see what social media accounts go silent over the coming days and weeks.
> from one Yandex data center ASN
This is only one of two extant Yandex ASN, not an ASN specific to one geographic region/datacenter.
You are replying to a war propaganda post with a heavily editorialized title. Of course it's swimming in bad faith engagement, that's how these things work.
An eye for an eye for the Russians.
I love western imperial propaganda though, and how HN so eagerly amplifies it. There’s something self-fulfilling about willingly becoming a foot soldier of the Empire.
Here's to a world where the West is monolithic, and may Iran, Iraq, Yugoslavia, Syria, Libya, Lebanon, Palestine be only be the beginning. And may every new crusade for freedom and democracy leave behind another Gaza, in pursuit of upholding human rights and civilizing the world.
The arms are sold(and often directly supplied) to Israel by the West. Israeli warplanes are fueled during attacks on Iran by German and NATO tankers. Israel's enemies have suffered decades of sanctions, economic strangulation, and demonization, with the intention of destroying those countries and nations - a crime beyond comprehension for the imperial foot soldiers. A foot soldier of the empire might still have a conscience, and that conscience needs soothing from time to time. So, the imperial machine declares "We condemn the cruelty" while continuing to expand its reach and devour everything in its path. The whole ME (and not only) is essentially a forward deployed ever expanding military base.
The point is that condemning a war in the ME is meaningless, because these powers do not want the war to stop, they just don't want it to be so blunt. They prefer economic strangulation, sanctions and demonization, they want it to be slower and less visible, obscured behind bureaucratic language, with its victims dispersed across societies, consequences unfolding over years and decades, violence concealed, and architects decentralized because it is not a crime of a man, be it Putin or Trump, but a crime of a system where no true accountability can take place. They do not need to bomb every school themselves, but rather enable minor military operations, undermine political and economic stability, and support policies that destroy a society's ability to survive. And the dozens of millions of excess and direct deaths due to these policies are meaningless, right? After all it's the school that counts - those girls had names, faces, futures, and we are rightly appalled. But the millions who die through sanctions, deprivation, and the systemic destruction of their livelihoods, they remain statistics. I mean, can you imagine the public appalled over sanctions against Cuba - what a waste of time. That's not prime time news, and it will never be. Because they work.
The deranged Orange Idiot and the bloodthirsty War Criminal are the ones carrying out what you describe. The rest of "the West" wants nothing more to do with them, and is currently attempting to avoid all the knives heading for their backs.
The war crimes I alluded to were carried out for decades under the guise of democracy and human rights. And those same knives were pulled out from the backs of the countries that I listed above. "The West" just wants to continue using the knives quietly, expand their military presence, and sanction the opponents to submission and destruction. The destruction and crimes against humanity in Gaza did not start with Trump, they have been going on for decades. You just dislike that now it's so blunt, you'd rather have their water poisoned so that they die out slowly and "without all the drama".
It's interesting (but not surprising) that so much traffic comes from the Netherlands and Singapore despite their sizes compared to the countries at the top of this list.
138 comments
[ 0.26 ms ] story [ 16.2 ms ] threadhttps://www.peeringdb.com/net/1751
Of course they have bots... That's how you index the web.
If we took Google offline I bet there would also be a reduction in bot traffic from them too!
Indexing is also probably a low priority task, so if half their datacenter capacity is taken offline, their bot traffic probably reduces by more than 50%.
Maybe they'd care if the account was using a stolen credit card and there was a chance they wouldn't get their money, but they definitely do not care otherwise for normal people like me. I'm sure that'd be different if it's a multinational or a country-level government body is reporting something, but I can't get those to do my bidding.
/s
Continue drinking the Google kool-aid
I've never noticed occurrences of such stuff, so I'm curious what I'm failing to key in on.
No. If you can't see it, and don't understand it's there, you cannot be explained to. You're someone with no palate asking how anyone else can know if a wine tastes good.
A sibling comment replying to you is using a familiar tactic, which is to try to shame you for asking for specifics, while simultaneously providing none and telling you that it's obvious.
Occasionally people do provide details and I learn something, but that's not the common case, sadly.
Just search for “Russia” on HN and follow threads. You’ll find lots of “what about West/Jews”, “Russia is fighting against NATO”, etc. Then open any western news outlet on YouTube and follow comments too.
Or is there other information?
Now there is probably also AI agent spam from google, but not at this level I believe. Also I am not fully clear what they count as spam, I doubt they would include google in this list, so do they include yandex bots here, if they come officially?
And most of those inside allowed this to happen over decades since even harsh dictatorships represent underlying population in many ways, many even cheered / still cheer on direction their country has taken, and non-trivial amount also like raiding neighbors for the kill, loot and money in general.
A stark warning to US if you keep letting things slide to evil direction, this is roughly how you end up over time.
Yes, but it's mostly the opposite tribe of the one you're complaining about.
Touch grass.
https://radar.cloudflare.com/bots/as200350?dateRange=7d
https://www.peeringdb.com/net/20950
I wonder what kind of fluctuation we will see in these bots traffic.
That would be awesome.
Edit: Here is some more info, they do have availability issues as is to be expected
> Because three of its primary facilities were knocked offline within four days, Yandex’s failover mechanisms broke down[1][8].
> Outages have cascaded across Yandex Go (taxis), Yandex Pay, food delivery, smart home platforms, internal logistics, and third-party corporate services hosted on Yandex Cloud[2][7]. Problems have spilled into neighboring countries using these systems (Belarus, Kazakhstan, Armenia, Serbia)[2][7].
Their Kazakh regions are available according to the dashboard, so it isn't as catastrophic.
Their managed DBs and data processing services are heavily affected though with no Kazakh redundancies from the looks of it.
If what you deduct were correct, Yandex would not be telling its customers to switch to competitora like VK Cloud. They even have a "priority onboarding code" called "CloudBro" for three competitors.
For additional strikes or to upgrade to ICBM-based payloads, you'll have to contact them to set up an enterprise plan.
It's hard to go to war now when your enemy can just use a few small drones to take out everything your citizens care about in a single day (social media, online deliveries etc).
And these things are practically irreplaceable in a war when you're sanctioned up the wazoo. Where are you buying racks of Xeons and DDR5 RAM and Nvidia GPUs?
https://cofense.com/blog/how-cloudflare-services-are-abused-...
https://www.heise.de/en/news/Report-Cybercriminals-use-Cloud...
Or maybe they would just want to get rid of the competition.
Pretty benign AS as far as actual exploit or malicious crawling is concerned.
[1] https://news.ycombinator.com/user?id=ericpauley
If so, it isn't particularly surprising that the line would go down.
EDIT: yes, it is. Looking at the same graph for all of Russia, and there's no discernible pattern:
https://radar.cloudflare.com/bots/ru?dateRange=7d
same for the 48h view:
https://radar.cloudflare.com/bots/ru?dateRange=2d
That is a drop in traffic specifically from Yandex bots (which includes their crawlers and other apps hosted in their DC)
Now compare it with Bot Traffic in:
* Russia: https://radar.cloudflare.com/bots/ru?dateRange=7d
* Europe: https://radar.cloudflare.com/bots/europe?dateRange=7d
* Worldwide: https://radar.cloudflare.com/bots?dateRange=7d
Dip is barely noticeable even in Russia.
Its like saying, "API requests are completely down" (and small disclaimer: only from your server, because your server is down)
https://radar.cloudflare.com/traffic/as12876
> from one Yandex data center ASN
This is only one of two extant Yandex ASN, not an ASN specific to one geographic region/datacenter.
https://www.peeringdb.com/net/1751
The other one is this: https://www.peeringdb.com/net/20950
And the 200350 also shows a near complete drop off in bot activity:
https://radar.cloudflare.com/bots/as200350?dateRange=7d
There is Hetzner for that.
All it shows is direct traffic against CF infrastructure.
...it's fucking datacenter, obviously it runs more than just "crawler bots"
Here's to a world where the West is monolithic, and may Iran, Iraq, Yugoslavia, Syria, Libya, Lebanon, Palestine be only be the beginning. And may every new crusade for freedom and democracy leave behind another Gaza, in pursuit of upholding human rights and civilizing the world.
The point is that condemning a war in the ME is meaningless, because these powers do not want the war to stop, they just don't want it to be so blunt. They prefer economic strangulation, sanctions and demonization, they want it to be slower and less visible, obscured behind bureaucratic language, with its victims dispersed across societies, consequences unfolding over years and decades, violence concealed, and architects decentralized because it is not a crime of a man, be it Putin or Trump, but a crime of a system where no true accountability can take place. They do not need to bomb every school themselves, but rather enable minor military operations, undermine political and economic stability, and support policies that destroy a society's ability to survive. And the dozens of millions of excess and direct deaths due to these policies are meaningless, right? After all it's the school that counts - those girls had names, faces, futures, and we are rightly appalled. But the millions who die through sanctions, deprivation, and the systemic destruction of their livelihoods, they remain statistics. I mean, can you imagine the public appalled over sanctions against Cuba - what a waste of time. That's not prime time news, and it will never be. Because they work.
Monolithic indeed.
I do that since years and the Internet keeps working just perfectly fine.
If you've got servers and serve "the west": just drop all these commie cunts countries entirely. Simply pretend they don't exist.
And nothing shall be lost.
https://radar.cloudflare.com/bots