Launch HN: Context.dev (YC S26) – API to get structured data from any website (context.dev)
I built Context.dev (https://www.context.dev/) to make it really easy to integrate web data into your products and agents.
Here’s a demo video: https://www.tella.tv/video/build-faster-with-context-dev-api...
Since it’s an API, here are the docs: https://docs.context.dev/quickstart.
You can send us a URL and get back clean Markdown, rendered HTML, screenshots, extracted images, etc.. You can also send us a domain and get company or brand context: name, description, logos, colors, fonts, social links, screenshots, style information, and related metadata. For more custom use cases, you can send a URL plus a JSON Schema and ask us to extract structured data from the site into that shape. For example, you might ask for pricing plans, product categories, office locations, support links, integration partners, or anything else that is visible on the public site.
The goal is to give developers the output they actually want. Raw HTML is rarely the useful thing; the useful thing is usually Markdown for a model, JSON for an application, a logo for a UI, or a structured company profile for an agent.
Before, I worked at Amazon and Sunrun, and co-founded StockAlarm.io & essense.io, both of which were acquired. Also, I built knifegeek.io, which scraped pocket knives from across the internet and listed them easily. The project is outdated now (coming back soon) but back then it hit the frontpage of hacker news and people seemed to like it: https://news.ycombinator.com/item?id=34604281.
Just before Context.dev, I built Brand.dev. The idea was that your software product should automatically know about your customer if they sign up with a corporate email. The API pulled brand data such as logos, backdrops, name, description, industry, and more from the public web and surfaced it to your product to integrate as part of their onboarding experience. That’s worth doing because conversion rates on onboarding improve dramatically when you go from “enter all this info” to “confirm all this info” (and there was never any privacy concern all the information is public).
That was a nifty niche, but the more customers used it, it became obvious that “brand data” was only one slice of a larger need. People started asking for things like screenshots, structured extraction, and LLM ready data. So I expanded to Context.dev, and applied to YC (got rejected after an interview), then kept going and re-applied at which point I got in as a solo founder.
People use Context.dev in more ways than I can list, but here are some: keeping context up to date on customer websites for chatbots - building beautiful brand assets/ads for customers - enrichment flows using agent harnesses like eve.dev - crawling customer websites into chatbot knowledge bases - turning GitHub repos into branded docs sites - academic journal and PDF crawling. There are a ton more examples at https://www.context.dev/customers.
We know that many crawlers are not behaving like good citizens on the web, and the entire space has a bad reputation as a result. At the same time, customers are not usually trying to buy “scraping”. They are trying to make a support bot work, personalize onboarding, enrich CRM records, generate docs, monitor leads, or let an agent research a company. There are lots of legit use cases. We want to satisfy those while being respectful of everyone involved.
We maintain a caching layer and avoid hammering websites. Customers can configure the cache, but if we find we’re sending too many requests to a url in a certain amount of time, we step in an...
71 comments
[ 4.3 ms ] story [ 74.2 ms ] threadshipping a bunch of new things soon which should make it clearer, but as of today yeah
AKA unusable for high value data.
I am interested in KnifeGeek though - looking for a good OTF (ultratech?)
Agents need clean/current context from the web, and this is the best way I’ve found to give it to them. The internet is clearly moving in this direction: companies are starting to realize their sites need to be legible to agents. Some are already adapting but many haven’t yet. Context feels like an important part of that transition
Yahia is a great builder. His pace of expansion has been impressive, excited to see where he takes Context.
brand data is a shockingly hard problem to get right
Of course, it's impossible to know for sure what was LLM processed or not, but some of your posts (like this one) have been getting classified that way.
> When Should I talk to sales? > Talk to sales if you need high-volume pricing beyond 2M credits/month, custom rate limits, SSO / SAML, SCIM provisioning, an uptime SLA, annual invoicing, an MSA / DPA, or a dedicated support channel. Reach us at hello@context.dev or through the contact page.
Would that this were the norm everywhere, rather than (say) a sales rep from Datadog scraping my phone number from who knows where to ask about my company's needs after I sign up for a free account on a whim :)
EG if I start passing in Linkedin pages what is your expectation of the result that people would see per profile.
EDIT:
Congrats on the launch seriously hard work, just wanting to understand your scraping stance more. I've worked with a lot of tools on this, didn't mean for my initial comment to be adversarial.
i got my first customer after 1 month, and then just worked like hell to make him happy even though it was a very large company paying me $99/mo at the time, they're still a customer too
since then, i just locked in on every customer until they were thrilled, and once we hit 50, word of mouth started to kick in, now we're at 300+ and get a referral every other day which is awesome
i think our community appreciates effort, and when someone goes the extra mile to make sure your problems are solved, you tend to remember that. it doesn't scale super well, but you'd be surprised how far it'll take you
> Reverse-engineers any website by doing a breadth search across every transport (JSON, WebSocket, WebRTC, GraphQL, SSE, HLS, PubSub), listing them all, and generating a typed JSON API that bypasses almost all bot protections — including Turnstile. I didn't include the ability, but it bypassed the most advanced ChatGPT + Turnstile. Built with self-improving Claude Code agents that rewrite their own instructions until fresh agents consistently succeed.
> Once connected to a page, it intercepts every byte of network traffic — then actively drives the page to surface endpoints that only fire on interaction. It types into forms, clicks buttons, scrolls, triggers modals, paginates, submits searches, and walks through multi-step flows, watching what each action produces on the wire. Every request gets captured with its method, headers, payload shape, and response, then classified by transport (JSON, WebSocket, WebRTC, GraphQL, SSE, HLS, PubSub). The result is a complete map of the site's real API surface — including the hidden endpoints that only exist behind a click — turned into typed proxy routes you can curl.
[0] https://github.com/adam-s/intercept
https://en.wikipedia.org/wiki/Semantic_Web
You’re on the playground
“That’s so gay”
Age: 12
It's a custom pipeline I made to address its shortcomings
Even within YC, there are many competitors that do pretty much the same thing:
- Firecrawl
- BrowserUse
- Browserbase
- CloudCruise
- NotteLabs
- Intuned
- Expand.ai
- Reworkd
And then you have the extremely well-funded web retrieval players like Parallel and Exa.
How do you differentiate to all these?
Another thing that might interest HN: AI crawlers come with negative side effects for website owners (costs, downtime, etc.), as repeatedly reported here on HN (and experienced myself).
Does Context respect robots.txt directives and do you disclose the identity of your crawlers via user-agent header?