HN Explorer

Show HN: Open-source alternative to ChatGPT Agents for browsing (github.com)

104 points by ElasticBottle ↗ HN
Hey HN,

We are Winston, Edward, and James, and we built Meka Agent, an open-source framework that lets vision-based LLMs execute tasks directly on a computer, just like a person would.

Backstory:

In the last few months, we've been building computer-use agents that have been used by various teams for QA testing, but realized that the underlying browsing frameworks aren't quite good enough yet.

As such, we've been working on a browsing agent.

We achieved 72.7% on WebArena compared to the previous state of the art set by OpenAI's new ChatGPT agent at 65.4%. You can read more about it here: https://github.com/trymeka/webarena_evals.

Today, we are open sourcing Meka, our state of the art agent, to allow anyone to build their own powerful, vision-based agents from scratch. We provide the groundwork for the hard parts, so you don't have to:

* True vision-based control: Meka doesn't just read HTML. It looks at the screen, identifies interactive elements, and decides where to click, type, and scroll.

* Full computer access: It's not sandboxed in a browser. Meka operates with OS-level controls, allowing it to handle system dialogues, file uploads, and other interactions that browser-only automation tools can't.

* Extensible by design: We've made it easy to plug in your own LLMs and computer providers.

* State-of-the-art performance: 72.7% on WebArena

Our goal is to enable developers to create repeatable, robust tasks on any computer just by prompting an agent, without worrying about the implementation details.

We’d love to get your feedback on how this tool could fit into your automation workflows. Try it out and let us know what you think.

You can find the repo on GitHub and get started quickly with our hosted platform, https://app.withmeka.com/.

Thanks, Winston, Edward, and James

9 comments

[ 2.5 ms ] story [ 31.2 ms ] thread
cahoodle
James here from the team! Let us know if you have feedback on either our cloud or open source repo. We want to push the frontiers for computer-use so that people can do less repetitive work.
chhxdjsj
Hi, great work congrats!

Does it use openrouter for model selection? Which models did you achieve the webarena result with? Are there any open source models which are any good for this?

armanj
Nice job. It's exciting that the quality is approaching human level, but still I think we are spending way too many tokens, and the automation speed-up isn't really worth the total token price yet (unless you have very high-end gpus and you don't care about the completion speed of your tasks)
esafak
I wish AI products competed on token efficiency.
h23bhati
This is awesome, biggest open-source browser agent?
nsonha
Tested a few agentic browsers such as genspark, fellou and comet. I found the vision approach less effective comparing to the dom-based approach, and seem quite slower too. Does it need a reasoning step to type an url into the address bar?
gotten09
this is pretty awesome, on the cloud env though I got the error: Error: AIProviderError: AI provider failed to generate text. Timeout while downloading https://playmatic-screenshots.s3.us-west-2.amazonaws.com

Also the task I gave it this was the result:

I was unable to retrieve any live fare data because both airline sites became unworkable in the remote session (xxxx selectors would not stay open; xxxxsearch could not be completed before the session ended). Below is a blank comparison table you can fill in once you gather the prices manually:

is that the current state of best in class computer use agents? or is more of a we need to modify it until it is good for our use case?

trying to provide helpful feedback and honest curiosity, this is awesome work

pants2
This is great. Will it solve the three biggest issues with ChatGPT agent?

1. Proxy support for sites that block the user

2. Browser extensions support for uBlock, password managers, etc.

3. CAPTCHA solving