HN Explorer

Show HN: Reverse-engineering web apps into agent tools

96 points by pancomplex ↗ HN
Hey HN! We built a browser-based agent that runs inside an authenticated web app, watches how the app calls its own APIs, and automatically turns those into agent tools. You can think of it as an auto-generated MCP server that self-updates as the host app changes.

The result is a skilled AI assistant that actually integrates deeply with any product (not just chat and RAG) with minimal effort.

Check out these short demos below that show the agent in software you're probably familiar with:

- Jira: https://demo.frigade.com/hn?skill=jira

- Spotify: https://demo.frigade.com/hn?skill=spotify

- Hacker News (lol): https://demo.frigade.com/hn?skill=hackernews

- Full Demo: https://demo.frigade.com/hn?skill=full-demo

As you can see in the examples, you can do way more (and faster) than what you normally would be able to via point and click. And we never even touched the source code of these products!

Why do this?

In an ideal world, every application has an MCP server or an easily-digestible API available for AI agents to feed from. In practice, we found that even very modern software tends to have a spider web of confusing APIs and services that AI agents simply cannot use out of the box. Security also becomes a huge issue as applications have different (often homebrewed) standards for how endpoints are secured (JWTs/cookies/mix of both). Finally, having an actual browser agent go in and use the application on behalf of the user (i.e. computer-use), is simply too brittle, slow, and burns a lot of tokens.

We took our existing browser agent that’s already trained to use and learn authenticated applications, and added an extra step that automatically turns the app’s authenticated APIs into "recipes". A recipe is a mix of the following:

- API endpoint + method

- Authentication method (and how to retrieve refresh auth tokens/cookies)

- Response schema

- Input schema (for POST/PUT)

- Human readable description of what the tool does

Putting it all together, these become reusable tools for LLMs, all without writing or maintaining any code. Even if the APIs change our agent figures this out and replaces the recipe for the tool with the updated version.

Adding tools to an AI agent becomes super simple this way:

- Our agent trains on the app and builds the recipes

- The app owner enables discovered tools from our dashboard

- The agent can now take actions on the user’s behalf directly inside the application. For instance, saying something like "invite my teammate to my workspace" would securely call the existing API endpoint for inviting users without proxying or relaying through a third party.

Of course, there's a ton of edge cases you run into when you try to do this - every application is intrinsically different despite how many "standards" exist. Fun fact: graphql was by far the worst API to work with in standardizing the recipes.

Looking forward to your feedback/comments!

33 comments

[ 11.9 ms ] story [ 895 ms ] thread
robszumski
Really cool. It would be interesting to see a demo of an app that is clearly more bespoke, like your Tesla account, online banking, movie theatre ticketing, etc.
[dead] no7z
[flagged]
zenniskayy
Very cool project. Looking forward to trying it.
west_subject
Is this opensource ? like the tool itself ? in it's entirety ??
akash91
Which LLM model is it using?
hajimuz
This is really helpful. I usually manually do this with chrome dev tools, it’s kind of tedious especially the header and cookies handling.
pancomplex
Agreed - and especially when the API changes..
spzb
You want me to run a closed source, LLM agent inside my browser with access to authenticated API endpoints ?! Thanks but no thanks.
pancomplex
No definitely not!

Our customers run the training on their own applications using a demo or staging environment. Then they install the in-app agent you see in the demos and turn on the tool calls they like. All the API calls are executed client side and never touches any of our servers.

[deleted]
(comment deleted)
nixus76
Very cool! Does it require some kind of scraping of a third party web app, like clicking through with a browser agent? If so, how can I be sure it does not delete my account or subscribe to another plan or make some other destructive actions if I allow it to do that with my authed acct?
dodobirdy
Now this is something truly agentic.
arjunchint
I am not following a couple of things:

- you sell to websites an in-app agent

- why not just have them give you API spec, why reverse engineer their APIs?

A bit longer term, would you see yourself competing with WebMCP then? Because the website can just expose those APIs to any browser agent

[deleted]
(comment deleted)
teravor
it's more useful when you don't need permissions from anyone.

1) get firefox MCP

2) visit target site

3) point GLM 5.2 to MCP (maybe grok 4.5 will also work here? haven't tested their guardrails yet)

4) instruct model to create userscript that augments/breaks whatever you want, or just break it with devtools for the one session.

between 3 and 4 you may want to coax the model into instrumenting the client to understand it, write a report/skill for future sessions etc

you can also use the above to create a custom client or just a bot using web API. you can also easily break bot checks without puppeteered browsers now: "the above script aims to determine if it's executed on an allowed web browser, map out all its interactions with web browser APIs to fake adequate responses while using the Deno runtime". it's generally better to augment Deno to pass for Chrome and fool all the checks and their future versions then to just break a single check that may change soon.

quarkcarbon279
I'm so surprised you didn't add your agent on your own website? And the same fundamentals of a browser agent why you can't achieve everything with APIs alone apply here too inside a website. What's your take?

We have been building in the space balancing both with priority to GUI tools currently building on top of the great GUI experiences of a product, rtrvr.ai/rover

hoppp
How do you make sure you are not violating terms and conditions if they forbid injecting third party javascript into the website explicitly?

Does your LLM ingest terms and conditions too?

In some cases in the past, finding and using auth tokens in the front end has resulted in hacking charges, in those cases alleged hacker found an api key in the front end and directly accessed the database, which resulted in a jail sentance.

How do you solve this issue that some companies may become hostile if injected code extracts auth tokens and accesses backend?

enos_feedler
Many years ago I wrote a simple script to periodically pull my aeroplan miles from Air Canada using an extracted auth token. I was building my own mobile app to track all kinds of important numbers in my life. One day I woke up to my air canada account being nuked without access. I was on the phone for 3 days talking to many layers of support. At one point they were not going to re-enable my account and would lose 400k points. I was devastated. Thankfully i was able to get it back eventually. This whole era of using LLMs to manipulate services on the client is not going to end well. We need proper business-level support or its just an irresponsible hack you are letting loose into the wild.
pancomplex
We only do this with consent from the application owner - the use case is typically companies who don’t have an AI agent yet who want fast path to get one.

This is the process we usually do: https://frigade.com/how-it-works

ikuminw
The self-updating recipe concept is good. Way better than maintaining hardcoded API specs that break every time the app updates.
throwawayk7h
This is going to wind up with remote attestation being implemented into web browsers to prevent this exact thing (and conveniently for google, this will finally end adblock)
gauravs19
Interesting concept, any performance or utilization impact?
pancomplex
Luckily no. The calls are executed client-side just like they would if you were using the UI.