A stateful browser agent using self-healing DOM maps (100x.bot)

Is this able to load for anyone?

Maybe this is a lack of understanding on my part, but this bit of the explanation sets off alarm bells for me:

> Under the hood, we're building a client-sourced RAG for the DOM. An agent's first move on a page is to check a vector DB for a known "map." ... This creates a wild side-effect: the system is self-healing for everyone. One person's failed automation accidentally fixes it for the next hundred users.

I think I'd like to know exactly what kind of data is extracted from the DOM to build that shared map.

This is, as far as I understand, self healing ONLY if the name of a CSS class changes. Not for anything else. That seems like a very very very very narrow definition of "self healing": there are 9999 other subtle or not so subtle things that can change per session or per update version of a page.

If you run this against let's say a typical e-commerce page where the navigation and all screen elements are super dynamic — user specific data, language etc. — this problems becomes even harder.

Couldn't you solve this by having the agent do a first pass through a page and generate a (java)script that interacts with the interesting parts of the page, and then prepend the script (if it's short enough) or a list of entry points (if it's not) to the prompt such that subsequent interactions invoke the script rather than interact directly with the page?

Neat approach, but seems like the eventual goal of caching DOM maps for all users would be a privacy nightmare?

Asking here because it seems related: I'm trying to use cursor to work on a webapp. It gets frustrating because vanilla Cursor is "coding blind" and can't actually see the result of what it is doing, and whether or not it works.

I ask it to fix something. It claims to know what the problem is, changes the code, and then claims it's fixed. I open app, and it's still broken. I have to continuously and way to often repeatedly tell it what it broken.

Now, supposing I'm "vibe coding" and don't really care about the obvious fact that the AI doesn't actually know what it is doing, it's still frustrating that I have to be in the loop just to provide very basic information like that.

Are there any agentic coding setups that allow the agent to interact with the app it's working on to check if it actually works?

“One persons map fixes everyone else’s”

Hm somehow I feel like this is a giant step in the wrong direction.

This tool seems relevant to my interests, but I gotta say I cannot figure out how to use the extension.

It seems like I'm only able to use the pre-existing/canned workflows that are provided under different "Persona"s? And there's no way for me to just create a new workflow from scratch for my specific use case.

Am I missing something obvious?

DOM has clearly gotten to the point where it's no longer maintainable or a net benefit to the web.

is there an open source version of this in github? i think i've seen something similar.

one off putting thing about installing the extension is all the reviewers seem to be Indian and I've seen similar patterns across Google Reviews where there is a flood of reviews from Indian users and they are almost always fraud or some weird scam

not saying this is the case here but whenever I see a bunch of reviews from Indian names, it automatically makes me trust whatever service or product less just fyi.

Is Agent4 open-source? I'm only installing OSS browser extensions for some level of verification.

I vibed something like this for markdown extraction just a week ago: https://github.com/promptware/readweb

Opensourced it just now.

More specifically, it works like this:

  suggestPreset: HTML -> Preset (via LLM)
  applyPreset: HTML + Preset -> Markdown (programmatically)

Where preset is:

  type Preset = {
    // anchors to make this preset more fragile on purpose.
    // Elements that identify website engine layout go here.
    preset_match_detectors: CSSSelector[];
    // main content extractors
    main_content_selectors: CSSSelector[];
    // filter selectors to trim the main content.
    // banners, subscription forms, sponsor content, etc.
    main_content_filters: CSSSelector[];
  };

suggestPreset uses a feedback loop that enhances + applies preset until the markdown is really clean

So to make sure I am understanding this, even though a site update its selectors weekly for example like LinkedIn, your automation agent would still continue to work.

But if a website changes its UX and your recording no longer works then it will fail?

Working in the browser agent space myself, although you save on cost with these repeatable recordings the true disruption of browser agents is using one prompt on thousands of websites without having to worry about maintenance at all

This may not be relevant but it also can't bypass many CF protected websites. Even completing the checkbox challenge doesn’t resolve the issue, it results in infinite redirects to the same Turnstile page. I believe this isn’t an LLM interpretation issue but rather problem with how the browser requests are being treated by security endpoint mechanisms.

Why not consider the cosine similarity between elements and focus on developing better feature vectors? Possibly models trained with a focus on DOM semantics and graph structure?