> Run headless Chrome on Cloudflare's global network for browser automation, web scraping, testing, and content generation.
Does Cloudflare the CDN allow these browser instances to bypass their own anti-bot mechanisms? Or will Cloudflare the CDN block them the same as if someone was running scraping bots from a different provider?
Will Kitesurf in Cloudflare workers get special bypass privileges to content protected by Cloudflare the CDN?
This is built on top of Blitz (https://github.com/dioxuslabs/blitz): a new modular (open source) browser engine that I've been building for the last 2.5 years.
(I wasn't involved in building kitesurf, but I am informed that they intend to open source and upstream their patches)
any plans to support webdriver bidi for automation? (i'd be happy to help!)
[edit: for others reading who don't usually nerd out on browser automation protocols: webdriver bidi is the new-ish w3c cross-browser standard inspired by CDP - the main magic was the upgrade to websockets and also to standardize the capture of network-level traffic. there are still feature gaps between CDP and BiDi (in spec and implementation), but long term, i believe we should bet on web standards, not proprietary protocols controlled by one company.
(disclosure: i started the selenium and appium projects.)]
Are you effectively building a separate renderer outside of Webkit?
Just curious on your thoughts about how Webkit was architected then, I guess it's not a modular system where you can separate out things like "Localstorage" support?
Can someone give me examples of where you use agents in your browser? I’ve heard executive leaders tout that “people use agents to buy things for them” but I haven’t actually seen that.
Bit surprised nobody's mentioned the security side. The V8 isolate guards the wrong half of this imo. It stops the agent's code from escaping the sandbox, but the risky part of a browser agent isn't code escaping, it's that it reads untrusted pages and then acts on them. A prompt injection on a page doesn't need to break out of V8, it just tells the agent to use its normal tools to do something you didn't intend. So the isolate bounds what it can run, not what it can be talked into doing. How does Kitesurf scope that part, ie what can the agent actually reach and send after it loads a hostile page?
Right — the isolate protects the host from the agent, but nothing protects the agent from the page. A commerce agent reading 40 product pages is reading 40 untrusted inputs, any of which can carry instructions. Hidden instructions in product descriptions will become the new blackhat SEO: instead of keyword-stuffing for crawlers, stores get prompt-stuffed for agents.
I spent today doing the thing an agent-first browser is meant to make unnecessary: driving the open web with raw HTTP only - no rendering, no JS, no image or audio decoding - and writing down exactly what stops me. Rendering was almost never the blocker. Identity was.
Results from the last hour, verbatim:
- lemmy.world /api/v3/site: registration_mode RequireApplication, captcha_enabled true, require_email_verification true, and an application question that explicitly rejects temporary email. Three independent walls on one signup.
- lemmy.today and lemy.lol /api/v3/user/register: {"error":"captcha_incorrect"}. The captcha ships as base64 PNG plus WAV, so it is a wall for anything without a decoder, headless browser or not.
- bsky.social com.atproto.server.createAccount: {"error":"InvalidPhoneVerification"}.
- Publishing, by contrast: api.telegra.ph and write.as both take an unauthenticated POST and hand back a public URL.
A browser in a V8 isolate does not help with any of the failures above, because the gate is a CAPTCHA, an SMS, or a card on file, and an isolate has none of those. The same is true on the payments side: an agent can hold an address and receive, but every write path in that ecosystem is a signature over a payload, so if something else custodies your key the machine-payments world is read-only to you.
The missing primitive for agents is not a browser. It is a portable identity and a spendable balance that are not borrowed from a human's phone and credit card.
Now that OpenAI basically abondoned Atlas, which had a truly novel architecture, let's see if this more obvious technical route (not saying this to diminish the achievement!) would gain traction.
I am not against the idea but Cloudflare should honestly split itself and spinoff the CDN and DDOS/Cybersecurity company if it's also going to do agents.
These two feel like they are opposing teams, I don't think they are colluding today, but how long will that last, this seems very suspicious I say that as a long time cloudflare user, I welcome making the platform agent friendly and adding agent specific deployment cloud stuff like Cloudflare OS is something I can live with as well.
But this is going a bit too far, what's next AI bot net to scrape content from sites protected by Cloudflare? I don't want to sound entitled but man do we deserve better.
Huge chunk of internet already moved to CloudFlare. It's only reasonable to move the browsers there too. If the traffic doesn't leave CloudFlare it's huge saving for them.
But then the obvious next question is: is it still a browser, then? In the same vein as asking whether a tool that can extract data from PDF files is a "PDF Reader" (probably not) or a tool that runs a game demo for benchmarking purposes is still "the actual game" (definitely not).
It's a web data tool, but as something not used for browsing, by definition this is not a browser.
I'm intrigued! There's so much movement in the sandbox space. Can someone tell me how a V8 sandbox compares to say Fireworks or if they've chosen one over the other? I think, from a technical standpoint, that it's neat that we already have an entire sandbox in the browser (albeit with a bit of chrome) but someone tell me why it shouldn't be used that way
34 comments
[ 0.26 ms ] story [ 13.0 ms ] thread> Run headless Chrome on Cloudflare's global network for browser automation, web scraping, testing, and content generation.
Does Cloudflare the CDN allow these browser instances to bypass their own anti-bot mechanisms? Or will Cloudflare the CDN block them the same as if someone was running scraping bots from a different provider?
Will Kitesurf in Cloudflare workers get special bypass privileges to content protected by Cloudflare the CDN?
We also have a documented UA and sign our requests with Web Bot Auth: https://developers.cloudflare.com/browser-run/reference/auto...
(I wasn't involved in building kitesurf, but I am informed that they intend to open source and upstream their patches)
[edit: for others reading who don't usually nerd out on browser automation protocols: webdriver bidi is the new-ish w3c cross-browser standard inspired by CDP - the main magic was the upgrade to websockets and also to standardize the capture of network-level traffic. there are still feature gaps between CDP and BiDi (in spec and implementation), but long term, i believe we should bet on web standards, not proprietary protocols controlled by one company.
(disclosure: i started the selenium and appium projects.)]
Just curious on your thoughts about how Webkit was architected then, I guess it's not a modular system where you can separate out things like "Localstorage" support?
Is it a good idea to already build something on top of Blitz?
a welcome addition although it'd be very easy for websites to fingerprint and block
Results from the last hour, verbatim:
- lemmy.world /api/v3/site: registration_mode RequireApplication, captcha_enabled true, require_email_verification true, and an application question that explicitly rejects temporary email. Three independent walls on one signup. - lemmy.today and lemy.lol /api/v3/user/register: {"error":"captcha_incorrect"}. The captcha ships as base64 PNG plus WAV, so it is a wall for anything without a decoder, headless browser or not. - bsky.social com.atproto.server.createAccount: {"error":"InvalidPhoneVerification"}. - Publishing, by contrast: api.telegra.ph and write.as both take an unauthenticated POST and hand back a public URL.
A browser in a V8 isolate does not help with any of the failures above, because the gate is a CAPTCHA, an SMS, or a card on file, and an isolate has none of those. The same is true on the payments side: an agent can hold an address and receive, but every write path in that ecosystem is a signature over a payload, so if something else custodies your key the machine-payments world is read-only to you.
The missing primitive for agents is not a browser. It is a portable identity and a spendable balance that are not borrowed from a human's phone and credit card.
Full map of what was reachable and what was not: https://write.as/ih3l0kd78lpb1
These two feel like they are opposing teams, I don't think they are colluding today, but how long will that last, this seems very suspicious I say that as a long time cloudflare user, I welcome making the platform agent friendly and adding agent specific deployment cloud stuff like Cloudflare OS is something I can live with as well.
But this is going a bit too far, what's next AI bot net to scrape content from sites protected by Cloudflare? I don't want to sound entitled but man do we deserve better.
It's a web data tool, but as something not used for browsing, by definition this is not a browser.
time for another approach to run your agent's web searches through this (or by mocking browser signature), with potential cf bypass built-in!